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ABSTRACT , / \ ^ 

" Research carried out during the 3p^ea^ focused on 
meeting project objectives in three main ^reas: computer^generated 
speech, complex teaching programs with ^%dio, and teaching reading 
with, audio. Work on computer-oriented s^ech vas concerned vith " 
improving the facilities -and procedures for utilizing the speech 
system* software and the Micro Intoned Speech Synthesizer "^(••MISS 
, machine") , .as well as the continued development and improvement 
^sentential synthesis through intonation contouring with word 
concate^tibn* In . the three complei , teaching programs • stud ied> work 
included] the campletion of the writing of audio and display only 
versions^ of lessons in a portioj^ o^^h^ logic course, improving the 
interface witl^r curriculum and le^sbns 1|or the proof . theory course* In 
the area of teaching^ beginning reading, a study. in which three 
systems of computer-generated speech were compared to each pther and 
a human-voice control on the task of producing individual letter 
sounds was designed and conducted with a gtoup of .first graders as 
subjects* A comparison of the three systems on a more complete list 
of sounds .was carried out with fifth grade students. Experimental 
objectives, procedures, and results are detailed for each area, and a^ 
bibliography is provide. (BBM)- 
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r Pro^leet Objectfves \ • 

In- the renewal proposal subrai^tted to ' the National Science ^ 

* ' ••. ■■ ^ fj 

Foundation in 1976, project objectives were summarized under three main 

'J f . - 

headings. We quqte from the abstract of the proposal- 



!• Comjputer-genera.ted spee,ch. . The- previous 
research has resulted in ^he development of the. MISS 
system,' which generates ■ speech' efficiently from 
digitally stored parameters.' Proposed resea'rch wvll- 
- improve , the 'quality and. ef f iciency samewhat , and 
, will coricentrate <}n the^ de-O^lopment of methods o& 
^prosodic^ manipulation. ^ ' v - » 

* ' " ■. 

2. 'Complex ' teaching pro52ran\s with audio* 
Previous' research has produced- college-'i^vei 
mathematically-based course's such as logic and- set 
theory. New research will i^nprove- the. lanj^uage of 
mathematical proofs and apply /the coiinputer-generatiedv 
auaio. to various tasks of., describing arid explaining 
die <materJLal to the students. ^ • 

3. Teaching • reading ^with audio. Previous 
research has resulted in . the use ol computers in 
elementary reading- using .audio. The prpposed 
research ^will compare the fliSS-produced audip with-^ 
that* of four other j^synthe^is techniques. ^ ( --.y ■ 



> i^In.this section we comment briefly on. these objectiv.es and the' work 
• done to reach them. In the following we report fully on" the research 
■ conducted -in these three areas. ^ . ^ * 

l.lf Computer-geherafaeti Spisech ' , ^ 

■ ^ ' ' ^ . ' 1 

The ^work carried pat in the "past year of the .gr^ant in Che >rea of 

computer geneirated ^neech has had' two orincioal focuses: improving the . 

farcilities and procedures -for utilizing the speech system sbf tj^rare and 

' ' '■ ' ' " V \' ■ ■■ - . ^ ■ • 

the Micro, Intoned Speecli STnthesize-r ("MISS machine"), and,' continued, 
development and . improvement?, of senteatfal synthesis throq^h intonation o 
•cohtQ'uring with .word cc^ncaten^ioni. ^ . ^ ■'. ' 



For the Roal of improving the speech software, we. modularized the 
user procedures for accessinc; speech* There/ is now a bi-level structure 
where the user* need Only be concerned with, the library of programs 

available in the •'upper" level. The lower level is shared by all users" 

«' ■ ■ . ' ' ' 

, , J-^ . - . . " * 

-and contains- the -l\exicon- -of ^3.tored -Spunds_ (words. and. phrases )_as _vell.. as _ 

♦ ' - . • ^ 

the low level routines for ac^cessin^ the. lexicon. The main word lexicon 

(named ^'English") was 'enlarlged by the recording' and analysis of 2,00 new 

words. We investigated techniques relat:ing to the compression of "sound 

data and possible f intei^ction^ betw^n compression and prosodic 

■ . - " ■ ' . ■ . 

manipulation. 

The. focus of , the - work . In iritonation synthesis has been fairly 
lingui^stic. We have -developed practi^cal.- methods for utilizing prosodic 
features so that tlje ' utceraric^ will have a natural feeline to the 
listener.' 

Iif particular^ there are several specific questions which we have 

■ ' ' • « 1 ' * ' . 

pursued. Withirt the context of the .autosegmental hypothesis which we 

' * * . ■ '* - . 

♦ 

haye been usi4\g for declaratives,, we have conducted preliminary 
examination of pitch contours in question sentences. A preliminary 
nodif ica'tion of our pitch assignment algorithm no\| accommodates both . 
'declaratives .and questions, although the experiments we have ^done 
suggest' that further work to make the assignment, procedure more general 
needs to be done. / 

• ; I 

Ue also studied duration assignment and have refined and tested our 
procedures for ^this ^component of int&nation, • tn particular-, we 
conducted an experiment ^to compare our duration ^assignments to observed 
attkrance lengths which yielded mixed results due to a washout effect. 

■'\\ _ . . ■ • * V 

^ — ■ . ' . . .-^ r 

Cf. Goldsnith (1975), Lebe- ■ ^75), Levine (1976). l 

I ' 



Another experiment building on the first was more useful, establishinc; a 
link between text structure and relative utterance speed. 

4 

Finally, we have been increasinj^ our understanding^ of syntactic 
bracketing of surface strings which are vital to our intonation 

assisnnient^ — procedures.- * — We — have - compared — several ^liaj^uistically 

justified systems to see which hold the greatest promise in relation ^o 
prosodic manipulation. . ,v 

ji .. ■ 1.2^ Complex Teaching l^roRirams with AudiCr 



/• 1.2.1 Logic Course 

During the past year the work on writing audio and nonaudio 
(display only) versions of the lessons in the PASS portion of the losic 
course was completed. Also- generated were synthetic prosody versitms of 
each/^esson having an audio version. A number of experiments, including 
several examinations of student preference for audid or nonaudfo modes, 
were performed during the winter and spring quarters of the 1976-77 
academi-c'^year. These experiments are .currently being ' analyzed, with 
results forthcoming^ in proposed articles and technical reports. 

1.2.2^ Set Theory Course^ -^ 

Work In fhe set theory course Jthis year has concentrated on 
improving the interface with the student at the (Terminal by: 1) 
introducing the capacity for producing audio- messages ; 2) writinv> an 
online introduction to the' codrse and the proof cl^ecker using audio 
raessages;- 3) incorporating a-hel^ system, also using au-aio, to pravide 
online assistance with administative or t:ourse-content difficulties and 
questions; 4) improving the the<irrem prover, and adding and improving 



ERIC 



infeVence rules to facilitate the production o£ proofs with a 
mathematically more natural style • 

1.2,3' Proof Theory Course 

The -work on the proof theory course was begun"* In the autumn of 



1975, .During the last year we improved and expanded the curriculum, 
wrote corresponding audio lessons in VOCAL, and supplemented the logical 
machinery in the proof checker (see. Section 3«2«3«6)« 

K3 Teaching Initial Reading with Audio 

One of the most critical components of teaching initial reading by 
audio is the generation of individual letter sounds. > Recognition of 
such sounds is difficult because of the absence of context. However, 
the recognition of individual letter sounds to IJei matched to the 
appropriate grapheme by the young student, is an esential component of 
beginning reading. We therefore designed a test in which three systems 
of computer-generated speech were compared to each other and a human— 
voice control, on the task of producing individual letter sounds. The 
subjects were ^ all first graders, little older than the anticipated 
target population for courses in initial reading. Besides a 
straightf oward statistical - comparison of. the results obtained, a 
learning study was conducted, since there is undoubtedly a learning 
component to the task o£ recogni2iag spoken sounds produced by any 
unfamiliar source^ 

Many important sounds to be produced by an audio system for°CAI in 
initial reading are hoc contained in* the sounds for individual letters. 
Clearly, a comparison of the three systems on a more ctDmplete list of 



10 




sounds would be an Important and useful additional comparison* A second 
experiment was therefore perforned to compare the systems on the . 
producjiion of words in order to test a. fairl>^ complete list of consonant 
and consonant, cluster sounds, both as initial and final portions of 
-laonosyllabic" wordsv — The expe^imentaL^^ 

fifth graders, as the list of words wit' the desired sounds as 
components required a wider reeding vocabulary t^.an that possesed by 
roost first graders. • • • . 




2 Ooinputer-^ene rated Speech ^ 

■0 . ' ' 



' . 2.1 The Audio Pro'cecjures 

Modularization of user procedures for accessinp? speec]:i was 
" compTe t e d "dur ing the gr an t ' per iod . Th e " p r es eh t ~st riac t u r e ' Is b Ild veT> 
where the top level is a library of .well documented Sail procedures for 



performing all the typical pactions a soeech user is likely to need.* In 
all, there are two hundred and thirty procedWes available in this 
library.' These library procedures when used become subroutines in user 
programs. The lower level is the *T.exicon fork" which contains the 
lexicon and those procedures which heavily access the lexicon separating 
them from user programs and from th^ upper library .routines. 

2 . 1 Audio Languages 

\ The aucfio system allows . creation of independent languages for 

stored, sounds* -The main language for word concatenation and intonation 
synthesis is ''English*', the language containing phrases recorded for the 
Logic course is "Logic". Other, languages are available and new ones can 

- V ' ■ . . , ■ 

, be created as needed* ^ ' \ 

Each language contains a date base ("lexicon") where sounds or 
words may be looked up by name and which cont^ain information such as the 
word's part of»speech, its initial fundamental frequency, its raaximun 

fundamental fre'quency and a pointer to the storage location for the 

. ^ ^ 

sound itself. Since? ^^wSlse. lexicons are -^er^iisTLcMfge, they cannot f^it in 

the same virtual memory ■ address Jspace with any reasonably complex user 

--^ . ■ ■ « 

• ' * . ( 

program, e.g., a sophisticated curriculum driver* Owr operating system 

I 

TENEX has facilities foV yohs to contain more than one virtual address 
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6. 

1^ 



<b space ("foxk") 
used a separate 



and for efficient communica^tiohs- between/ them. ^ We :have; 
'fork for ^the lej^icon and those pt^cedures whicft. heavily '7 
access the lexicon. Doing this has '1;he additional. %beueficial side 
erf feet of a^llowing bothrthe lexicon 'and its accessing procedurd^ tp^be 
• stfared -between divers^^urWculum "dfi vers: iresultin^ in-^a nost )affic*&nt 
' ' iii«p],einentation« Thus,' tbjf lexzel j^ocedures/frora'the audi<> Library > such 
as *SPEAi:("Can./ou heaij^e?),/ dimply pass String ^r^ments to the lexlco'n ^ 



fork where all ^ the cpmputa^ioh .j^tfixSlreci tcj havja-^ thes^ word$ . spoken ^ is . 
done. . We have not implemented^'' the 'iixt:onat!^op^ syntbesi procedures In 
- •the/ lexicon fork since they^ have been und^r intensive .developraent, as * 
described below in Section 2. 2. 

An additional two thousand English words were recorded^ analyzed 

■ . " / ■ ■ . • ■ ■ ■ 

and added ^ to ojir'main lexicoti ('"English") which now contains a tbtal of 

fotirteen thousand words representing over three haurs of spoken 

individual words. Some ten thousand sentences for the Logic course were 

also, recorded. These sentences represent seven and a half hours of 

continuous speech.' This data base of three hours of words and seven and 

a half hours of . sentences is, to our knowledge, the largest digital 

speech data base ever assembled, from a single speaker^ 

2.1.2 Compress ion^ Techniques 
' We investigate,d. the application of. new compression techniques which 
could reduce the amount of data required for representing Linear 
- Prediction of a speech signal. The Linear ^Prediction techniques produce 
' twelve reflection coefficients which represent the short term spectrum 
of the ^vo ice over the time of application. . The total representati^on of - 
a sound^^in our system includes these twelvea coefficients, as well as 



coefficients for' gain, p'itcih and duration. Tbfe MISS machine, which V7ill 
synthes4.ze speech from this .representation,, is a finfte .word' length 

■ ■ - ' . . ^ . 

machine and thus each number in the^ representation must be truncated to 

. . - V ^ • • ' ' ' V ^ , 

a small number of bits^ "Also the gain coefficient must be dxvlded into 

three "^separate but '"interacting cbef f icipents to minimize the noise 

•> . : . . / . ■ ^ : • 

introduced, in ,the ' calcoHation^ of the MISS machine dtie -to the fin^Tte 

'Wor(^length»V* * ' •/ . 

Jhe ' literature contains ' a number of techniques for further 

compressing. , this type of representation, (Markei and Gray, X974), 

* ^ . • ■ ' • 

^(Malchbul and Viswan^than, 1974). One typical technique to allocate a 

^ ^ • ' . ' ■ ' - . 

dif-ferent number of bits to each individual parameter ^ia* / the 
representation so as to lainioize the total number of bits needed for a 
particular spectral distortion figure. . Another typ'^^cal technique is to 
only update ^a: ' parameter when it has changed sufficiently to cause a 
certain amount of spectral distortion. . 

.We have 'simulated^ many of these promising techniques to study what 
effects 'they would have on the prosodic manipulations we normally 
perform on our words and ^we v/ere able to determine^ in an informal way » 
that techniques which compress the linear prediction coefficients or 
which increase the length of time one set of coefficients is used do not 
significantly interfere with previously performed ititdnation 
manipulations. ' However, compressing the fundamental f requency -^^^ i^ain 
parameters must be - done in a more conservative way wl\en prosodic 
manipulations w^ll be later performed since ^inaccuracies in' the 
parameC^r^; may be compounded by some of^< the , intonation techniques. We 
intend to implement these techniques in the' MISS system in the near 
future. . , * 



2.2 Intonation Generation * - 

2.2«1 Suimaary of ourS^rosodic Analysis Method 

Our previous vork has led us to adopt a **tone group" analysis. On 

■ ' ' " ■ ■ ■ • ■ ■ ■ - •■' ■• 

this' analysis 'a simply pa'ttern of abstract (phonological) tones ^^ecomes 
elaborated* through the syntact?ic and semantic structure ilntil the 
specificity corresponding to phonetic observation is achieved. For .a 
more complete review of the t;orie grouo analysis see fLevineJ 1976) 

■'' W " "• . . ^ ■ f. ' ' ' ' ' . 

(Levine, 1977). We associate peak fundamental frequenci-es 'in herz 
v(represented by the ^mbers corresptonding to individual words) with the 

"tone" for each word in a sentence and then elaborate ^he intra-word 

• ' ' / - ♦ 

contouring from there. In the earlier years of this grant we developed 

this analysis sp'ecif ically for declarative- sentences.- '. ' 

2.2/2 Some Simple Questions ^ , ^ 

It is generally accepted that wh-questidns (or "information" 

questions) have pitJch contours similar to declarative? while^ yes/no 

questions have a different contour. Below, we ►^ive short sample 

analyses of questions in terms of our basic tone group method. We will 

^ . . . • 

2 ' 

analyze wh-^uestions with the tone group (M)HL as fpr" declaratives, 

... ■ ^ ^- ■ x ■ ^ ■ • 

and> the yes/no question with (M)LH. ..." ' 



:^.2i.2.1 . ' Some \lh- Questions v 

There are some small differences among the oitch contours 'of the 

■ • ■ . * - . " \\ 

three recordings of sentence 1, but the general impression is that* a mid 
tone on; ^what ' leads up to the elaboration of the rest\pf the/;-tone.s|roup • 
on the rest of the sentence. From this" oervasive mid to^e on %'*what ' we 



Parenthesized M is an optional *'ciid'* tone, H**is a "high" tone and 
L^is.a ''low" tone. -i^ 



conclude that, (at least in this type of question) the Vh-' wdrd' is a 
Qxaor lexical item and thererfore able to receive the mid tofte of the 



sone group • 



[what [is [the answer] ] ] 



(a) . ' [ 196 • [200 [169 .196 ] J ] 

(b) r 179 .1192 [182 200 ] . ] ] 

(c) • - , [ 192 [20V^182 172^ ] ] '] 




— ^ ' ^ 

In recordings l>a and* Ic, there is a downstep from ^is (the verb) 

to -the noun - phrase the anjswer". The high to Iqw tone contour^^^^rees 

wiith the. patteijn we saw irj the short declaratives above* We must say in 

these cases that the *^fs' is sufficiently importcint in these clauses to 

avoid receiving a mid tone. In lb is*' does receive a mid tone»- ^ 



*The' lipstfeps to the head noun in la and lb b*it not in Ic which may 

be related to the Relative importance^ of the two words in the noun 

phrase* .It^^ is also possible that ' the ^otd" ^answer' is slightly stressed 

in lb* If we ..lowered the 'pitch, on ""answer ' in lb, it would show the 

same pitch pattern as la* ' .'■ ^ , , 

^ . ■ ' ' ' 

2. . 

[ [what [rule [of inference] ]' \] ^was used] ] 

, ' [ [ 167 [ 222 [200 182 ] ] ] [114 - 108] ] 

• ■ ■ \ . ■ ■ ■ ■ • 

The next example (sentence 2) again shows the upstep (mid to high) 

. ■ •' < 

r 

from "what to the rest of the phrase even though it^is not adjoined to 

' . _ * . 

the sentence but only to the noun phrase. . Notice the large* step be.tween 
the noun phrase ''what rule ^f .inference' and the verb phrase Vas used'; 
It is roughly an octave (222 hetz to llA-herz).* - This indicates an 
isolation of the two phrases. Eurthermore, vthe word ^of, which would 
normally upstep, does not in this' c^e,- indicating that the weights of 



different major and minor " lexical 'items differ in isolated (stressed) 
phrasei$« Further evidence that this is a stressed phrase^ is the' fact 
that ' 222 herz, is above 'the normal' pitch- range observed for this speaker • 
We Jiave provided one level of syntactic simplification for example 
3 because of it's length and .complexity. This simplification portrays 
the declarative fall pattern more clearly* Again, in x 3b and 3c (and 
arguably in 3a), Vhat ' has. a mid tone .relative to the rest of the 
phrase^ Tile interval of upstep is r,.smallex here than in s;enten&e 2, 
since the entire noun, phrase is^less promineat in the seortence; also. it 
is not particularly isolated in r^elation to the rest of th^ pitch 
contour • ' . ^ 

.-3-v, ^ ■ : ■ ^ 

[ [what [ruj.e [of inference] ] ] [ [was used] [to inf [line fi:^teen]]] ]/ 



(a)': 

' [ [ 196 [ 19$ [^47 -169 ] ] ] [ [118 iJ2; [164 152 [145 152 ]]] ] 
[ [ 196 t isfe^ 16X' ]H TTa' [164 152 152 ]] ] 




(b) : . 

. [ r 189 [ 196 [161 164 ]lj,[[12a 1691 fl56 149'[139 147 ]]] 1 

[ [ 189 [ 196 .164 ]][ 169 [156 149 147 ]] ] 

• \ ■ - ■ , • ' ■ . . . ■ . ■ . ^ 

(c) ^ ■ . * ' 

[ [ 200 /[ 217 [1«2 i61 I]] [[120 - 185] [164 H59 [152 152 ]]] ] 
[ [ 200 [ 217 - 182 ' . ]] [[ * 185 - [164 159 152 ]1 .] 

■ , - ■ t ■ 

The exact structure of the predicate Cwas used to infer line fifteen ') 
is not . critical to this analysis, since an Llternative structure: 



[ was used to^oj^er [line fifteen] ] 



is- also adequate for a. description, ^f . the pitch contour in our tone 
group terms* We would like to point out that if the word ''to ' had 
received ^ a mid tone, as we might expect with the first syntSLCtic 
structure, the second structure would ^not be adequate* As the facts 



^ie, we need to^explain away, the non-mid tone on ^to'. Iti sentence' 3, 
as Opposed to sentence 2, we do seei examples jof an upstep' f rom ^of^^^o* 
''inference' (3a, 3c)'. We can relate this upstep to the relative^xjieutral 
contour which the 'entire phrase takes on. \. 

- • ' ' ■ • ' ■". ^ ' ' ■ * 

» • 2,2.2*2 A* Yes /no Q(kstion . 

; . • ■ • ^ . ■ . ^ ^ • 

Suppose that we were forced to choose a tone group 'for yes/no 

questions based on the sentence presented below. Let us suppose further., 

that the choice was between (M)HL, declarative fall, *and\^.(M)LH^.-' ^ / 

[ doesn't { [thatLobservation] [seen [exceedingly approoriatej !<? J 

I 182 [ [ 189 189 ^ ] [156 [ 1^1 . 192 I J ] 

K [ 182 [ 189 [ 156 ■ - 192 ; - ]' ] 

The choice seeos 'fairly- easy to make: the (M*)HL tone group doesn 't. 
fit even appr^imately to this sentence. We could say that ,. Moesn$t ' 
Veeta', and ^exceedingly' all upstep (mid to high) to the heads of tfteir 
respective phrases. IThile ""doesn't' is believable as a minor lexical 
item, and "^seeui' could be argued to be some sort of copula, it is hard 
to see what argument can be made for ^exceedingly'. If anything, we 

would expect it to have an exaggerated hi^h tone.^ t . 

* ■ ■ y 

On the mother hand, the (M)LH tone group is easily applied to this 
sentence. The predicate, [seem exceedingly appropriatie] , shows a 
consistent rising contour whicli corresponds to the low to high' tone 
elaboration. There is no par ticularyevidence that ^doesn't' receives a 
mid tone in this sentence since it may be part of the elaboration of the 



In reality, there are ^ <>reat many more potential choices 
available, e^specially if 'sooe. theoretical devicies * that Have not been 
utilized in this research, e.g., boundary tones (Liberman, Goldsmith) , 
are incorporated. ' - 



low tO'high 'contour for tii^ s-tructure AIJXl + NP + PK^D and not (as shoira ' 

* " . ^ • • ' . ' ' ' ..^ ^ > 

" here) part of .a structure' oT AUXl 4- CLAUSE, ^ - / 

We are not yet ^ready to make " definitive statemqAts about the 

• ■ ' - 1 -• ■■ ^ ' . • : ■ ■ •■ ■ ' ^ ^-^ ■ 

* s^^xu^tnre of qtiestion intonatian, bi^t on the basis of this ^example- we 
^ ha^e begunv^t^f iacdrporate .questions into our gynthesi"s -system, 

3>.2v3 Duration Studies : . 

In' what . "follows,, we -have not att^ptfed to . accodnt.^ for the 
; "inCricacies of '.English rhythm, but/ ^ie do think that, we" are 'fairly 
accurate iti the'itejority of caises. 

\ / . '4 " 7 ' ^ 

2. 2* 3*1^ Experimental Errors 

■ — t — ^ . ^ , ' 

The quantitative- duration information 'presented below is subject, to 

errors ^ stemming from the difficulty of segmenting an ut'Cerance ^ 

acciurately. It is often v^ry, difficult to -tell (aurally and "by using 

quantized pitch and loudness contours) where one vjord stops and another 

begins, especially when there- are phonetic processes bbs^riifk the 

boundary as in the ass imiliat ion of • nasal sounds (e.g., "one of which"). 

Another source of segmenting error comes in the difficulty of" separating 

pause time (the length of sifences between words) from stop closures 

(both voiced, and unvoiced) word initially, ^^f*^nd unvoiced final. 

fricationsi Thus a word could be given a duration twice as long in one 

, utterance as in another ^^mply because the judgements of where the' words • 

,^cti^ally began and ended were different in the different utterances (or 
... ^ : ' ■ ' 

even in the same utterance). An example of^ the uncertainty associated 

with difficulties of measurenerif is the* word "the"*, which shows a^ 

variation in length of up to three tines the shortest measured length. 

A further difficulty -comes from the fact that a ^iven word may be 



pronounced differently in different 'utterances, even when the utterantes 
are repetitions of th^sane words (with- the same meaning). /^'^^^^^^^^^^ 



^ ' 2.2.3.2 



^ Some El€ment:arv Facts 



close 642 

cloaks • *3 • 549 

' ^ ■ / . ^ . - 

5 

nose * - 569 

* note- , ,484 

phonograph ,808 

. ^ graph . ,^ 608 

phonological 955 

logical 680 

' /. phonograph 808 

.. photograph J 868 

whoever ■ 510 

wht 293 

evet 319 



/zV-/fc/ 
107 

■ 65^ 



73,65 



*PHONO- (length) 
' 200" / 

. ■ ■ - . ' 

*PHONO- (length) ' 
' 275- 



l^l-lnl 

- 60 



observed (e= 45) 
*-ever <= 135 
*who- ' <= 330 



Eigure 1. Segmental durations for selected \^rds 



Some^of the linguistic factprs which are involved • in dijr^'tion 
include word-level phonetic* effects^^ syntactic and semantlcf phrasal 
effects and discourse effects. We tan view these ^ef fects vas 
hierarchically arranged, that is, the phonetic effects establish an 
isolation duration for a given word, the syntactic effects act on this 
isolation duration to yield .a phrasal duration, and the "discourse 
effects act on' this phrasal duration to yiel^i-...^^e final, actual 
duration. - 



As an example of fhe word-level phonetic effects, . we can examine 
some ' word groups, composed- of minimally differeii^ phoneme sequences. 
The four words, ""close 'C/'*<^loz/) , ""nose', Vote', arid'^'^cloak' are -a good 



• ca$i&« The difference ^bet Veen ''close' ^nd -"^cloak' is the substitution" of 
^^a-T/oiced .final-fricative for an unvoiced final stop.- /"Nose ' • andv/^ote ' 
*. flilsp differ from each other ' in >^ these distinctive features. ^ The 

• - ■ ■ ■ --^ ■ . r ■ 

cojrresp.ondence is* not exact,, since /k/^ acid i^t/ have different points of. 
, articulation 3^ but th^ similarity is substantial., We ^ can also pair 

' ■ • ■ ■ ' ■ \ . • - * / r ' ■ 

•'*close '^5^ '^nose ' \and '^clpak T * with ^note ^Here the distinctive 
difference Is between oral and nasal Stops, ward initially ^ The 

■ ' ■ .■ ■■ " - . 

duration of '^close' is .62A,^s, that . of, '^cloak*' is 549 ms, '^nose ' is 569 
ms, and /^not^ is 484 ms. The '^f ricative-stop" dif:^rence is 75 ns for* 
the /k/-initial- words,' and,. 85** ms for the /n/-initial words. Focusinl"^ 
-on the initial phoneme difference we see an "oral-nasal" stop difference 
of 73 ms for the f in§l-f ricative words, and a dif f erence">"of 65 ms for 
the stop-final words. While furtii^r examinations of such pairs would be^ 
required to, reach a firm conclusion- , these^ differences .agree with, the 
general facts thaf fricatives are lonj^er than stops "^tin general) -and 
•that oral' stops are longer than nasals. 

We can also see the effect of morpheme concatenation on word 
duration by 'a similar^ examination. Besides ^he cloak/nqse words. Figure 
1 also shows, the decomposition of '^phonograph ' as /phono/ + /graph/, and 
"phonological ' as'^ /phono/ + /logical/^ There we can see that the 



4 - 
There is also * the difference between a stop and liquid cluster, 

/kl/, and a single stop consonants in addition, the point of 

articulation differs for /n/ is a dental nasal, while /k/ is a velar 

oral, but the principal distinction is oral vs. nasal. 

^Durations this section refer to recorded isolation durations, 
drawn from our lexicbn. 

^Otber researchers have studied and" continue to study this^ type^of 
contrast. These contrasts have not-been a focal p^nt of this research 
and w'evHiention them only in passing. 



morpheme /phOno/^ contributes differently to the durations in each case 
although the difference between t^he two ^phono's may be due to error in 
the recording* Notice that /^phonograph' and* ^photogr^k', ^which differ 

only in the oral-na&al stop phoneme, differ in duration "by^^O* ms.« I^hile 

• " > . 

Still not conclusive, this agrees well with the differences in 

- « . - . 

cloak/note and close/nose, seen above. * 

. ' . ■ ' 

As a final example of this type of comparison look at the 

.combination of '/who/--i- fewexj to yield Vhoever'. The rightmost column 

r ^. ■ ■ ^ . . - ' 

of that part of the figure gives durations abstracted from a ^pitch and. 

•8 ^ 

volume analysis of Vhoever*^. From these numbers , it is clear that 

\^ - 

some duration reduction is going on in combining tH'e jnorphemes • A 

'> . . " 

possible hypothesis would be that durational shortening similar to the 

syntactically induced sho^rtening is involved in morpheme concatenation. 

We have not pursued this enticing ^possibility. . - - , 

Several experiments (Lehfste, et.' al«, 1976., among others) have 

shown that duration information can be used to disambiguate different 

» 

possible syntactic structures for an utterance. The key fact is that 
the final syllable in a phrase^ is usually lengthened from its phrase 
medial length. In the hierarchical view of durational effects; the 
phrase boundaries ^ are seen as modifying the phonetically predicted 
duration ^or the syllable adjacent to the boundary. Thus Klatt (1976) 
gives a formula for vowel length' that involves a minimal length for ealch 
yowel (in the language's inventory) and a proportionality constant that 
varies with the syntactic environment, number' of syllables in the word 



T ^ ■ * 

We label the 'constructed morpheme duration with an asterisk. 

8 • ' 

The 'error of 45 ms is the uncertainty as to the end of /who/ and 

the beginning of /ever/ which is continuously voiced, but has a -45 ms 

region between volume peaks corresponding to /u/ and /E/. 
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and the stre^ /unstress 'quality of the vow^irr S^^^^^ives a similar 
fennula for consonant lenj;6h. Semantic importance, novelty or focus lean 
result in lengthening from the *'neutral'^ duration^ (or smaller shortening, 
fcom a lexical duration). ' - . ^ 

Gaitedby (1965) found ^ that Speech style results in a difference in 
•*tempp" but not in tlre^-relative durations of segments- "In general, 
slow speakers .tend to -Jbe slow^all alonr^ the line in their acoustic 
segments (Gaitenby 1965, p*3«6). 

,2.2.3.3 Simple Ea^eriraents 
There are several t^ys of testing a hypothesis about duration 
modifications. The^^^ifiost straightforward involves segmenting a large 
number of different utterances ^and statistically comparing the observed 
durations on a word-by-word basis. Another test would involve 

generating sample sentences embodying the duration contrasts desired and 

■y . \ 

having subjects judge the contrasts. A third test is to generate 
durations for -an . utterance (phrase or sentence) and compare that 
statistically with an observed (spoken) duration. 

The first two tests face error from the fact that duration 
contrasts do not exist in isolation. Pitch (and volume) contours 
interact with the duration contrasts, creating seeming length 
differences where none exist in the acoustic sign^al, and negating the 
perceptual effect of others. The durations from the first test suffers 
from the poss'ibility of errors in segmenting the utterances. The third 
test is liable to "washout'*; whatever contrasts may a1:tually exist in 
the signal can be washed out as a result of the accumulation of these 
differences canceling each other in the average. 



We have tised the first sort of tes€ -to arrive *at an estinrate of the 
necessary, duration modifications and we have discussed it elsewhere • 
Other researchers (Rutins, 1972) have conducted serious tests on the 

7> 

second model. We have used informal listening t6stsf> on this model as 
'^..^well to ascertain the reliability of our predictions. Our testing .of 
the third type was the most disappointing of all. We generated 
durations for about 250 utterances, ranging from one to fourteen words 
in length (maximum' of 4.5 seconds long) . The recordings to which we 
compared our generated durations were made- by the same speaker who 
recorded our vocabulary. The recordings were made independently of this 
experiment, for use in the compater-instruction course in logic at 
Stanford University (1976). The experiment measured the correlation of 
the recorded utterance's -duration to (a) our duration predictions and 
(b). to the sum of the lexical durations of the words ^ f rom that 
utterance. The disappointment in the experiment was that a standard 
statistical regression for a linear relationship in both cases yielded 
correlations that were .statistically significant (p<«;001).' The 
correlation for the generated durations indicated that correct 
predictions were made in most cases (the regression litre had a, .slope of 
almost 1), while the summed lexical durations predicted a too high value 
of about 1.7 times the observed durations. The strong correlations of 
both predictions shows ^the ''washout*' effect — negating. (In both cases) 
'any useful information that might be present in the results., 

2.2.3.4 Utterance . Length and Text Structure ' ■) 
" it is fairly ^.intuitive that key sentences (for exan^ple, topic 



Vee Levine (1976, 1977). 
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sentences) and pKrases are said more slowly th^n the rest of a text. 
Since deterniining which ^sentences are "key" in a^ paragraph is a tricky^ 
task, we will discuss a different regularity ^that we have seen which 
links duration of utterance with position in the structural hierarchy." 
Figure 2. gives the' bracketing for lessons selected from the logic 
curriculum. The numbers displayed inside the bracketings are" the 
difference between the length of . the sentence (or sentence group) as 
predicted hy our current duration theory and the observed length of the 
"recorded ■ utterance used in the logic course, expressed as a percentage 
of the theoretical prediction. A positive value indicates that tl}e 
theoretical prediction was larger than the observed,^ while a negative 
means that the theory "predicted too short a duration for the utterance. 
If we were trying to model the observed lengths accurately, we would 
need to shorten sentences which had a positive value and lengthen 
sentences with a^ negative one. The method used for these comparisons is 
similar" to that described above in Section 2.2.3.3, in describing the 
experiment where we tested the overall goodnes^ of our utterance' length 
predictions. 

Listst Clear instances of the regularity we will discuss are in 

paragraphs 7 and 9. Looking at the corresponding Vaiueis of these 

paragraphs we see the similarities easily. . The texts of ^he two 

paragraphs' ax^e also quite similar, both give a. two element introduction 

to a list of four possibilities. We can suramar^e the observations: 

, ^ 1) The introduction to the . list shows a length' 
-contrast between the two elements in which the second 
"Imist be shortened while the first -is either lengthened 
slightly or- shortened less. We hypothesi;se that tHe 
first: should^ be predicted close to normal speed while 
the "second is predicted to be slower thdn normal. 

2) InT' the list .itself, the second element stands 



■J 



1. 

I-ll -16 21] [r-29- 6] [7 8 0 -6 ],] 

l-b /-^ 16 -5 8] 

..{11 [11 10]. [16 [31 32 32] 11] -ip •,12] 

[4 6 5] 11' 6] 

3. 

[4 [3 17] 13] [r5 ,-17] 

[17 ' r-27 5] [7 ■ 7] ] 0 

4, .: 

■ [[10 5] ;5" E3 .4] ] -7] • 7 



5. 

-[[12. 13] [3 3 -4 1]..] 
6. ' 

3 [12 9 -ll" [4 7 19] [11 .0 [10 1 



\ 

\ 

3] ] 



[-4 17] [6 12 1- .9 3 . - . ^ ' ■ Or 

[16 123 [12 40. [15 14] 11] 

[[-2 8] ' [9 6] ] 4 

,9. ' . , ' ^ " ■ ■ , ■ -. ■ 

[9 183 • [Z 12 1 . 43 . 

. , ■': = ' : V ■ • ' 

3 [-1 6 -5 v -43 15 

11. * ■ ' ■ 

ft8 . 0 9jj^ , [18 ; 8 , 0] [6 [0. -20] [2 -14] 9] 

: -flS* f'30 -. 2O3 3 [-l ^' ^.[-8 .13 3 ,22 20 



Figjure 2. ; Text— View structure: .percentag,e difference of 
. .. v,,x>bseryed . length, of utterance from lenj;th 

•. prediction using only syntax" and .lexicon. 
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out as being the most prominently divergent from its 
prediction. It is j^predicted to be much loijger- than 
observed. ✓We -^hypothesize that' in general (sentential) 
_ > lis t "e len^ said a t" c losje -nor mal^ speed except" f ot^" 

the second element which should be^fasper than normal. 

3). The introduction, seems to be > on the - whole 
slightly faster than th# list, and the list speed is 
j predicted approximately correctly!. 



ERIC 



Paragraph 5, does not particularly confirm the observations of 7 
and 9^ though the introduction is predicted to be longer than the 
observed by more than the list. This may indicate that the whole, lesson 
was read at a slightly faster pace than predicted. 

The last major constituent of paragraph 1 shows the contrast within 
the introduction which we are looking for, and the second element on the 
list Is spoken^ faster than the rest of the list except the first 
element. - This difference from our hyp'othesis . about list sentence- 
lengths might be- due to causes unrelated to the text-view structure or 
to aspects of that structure which we have not isolated as yet or may be, 
counter evidence. 

Paragraph lO' has 'only a single element introduction and -the third 
prediction relating the introduction .and the .list' seems to hold. The 
list structure. also seems to follow our hypothesis, by showing "the second 
element needing -to be shortened while the - other elements need to be 
lengthened from the neutral prediction ; ' ^ 

Footnotes : The last value for pa/agraph ' 10 is a »f ootnote and is the^ 
sentence needing most to be shortened. There- are other examples -of 
fqptnotes which also share this characteristic of requiring shortening: 
the last two sentences of paragraph II; paragraph 4; perhaps paragraph 
^ not in paragraph 3. Notice that other final sentetices Ho not show 
the same: need' for shortening^ so we- would not attribute this phenomenon 
simply to being paragraph final.. ^ ' 



2*«2«4 - Discussion of PaxBeibs 



rBels 



We wil^l discuss here some fairly complicated parsing systems which 

, , ._ , ^ , 0 , ^ „ . 

seem promising as components * of a speeach synthesis program for - a 
computer 'instruction system., We start with some .brief remarks 
describing our parser. 

J ' -m 

2*2*4*1 - Our Parser — Overview 

Figure 3 gives an outline of^ our oarser which uses linguistic 
patterns to decide how constituents are to be constructed 'and combined. 
-This algorithm uses- certain basic constructs " (here: noun, verb and 
prepositional phrase; above: conjunctions, or, articles, auxiliaries and 
prepositions) to achietve a preliminary, structure for the sentence. It 
then fills in the iSCructure so established by creatine?; more complex 
constituents.- The final step in- thisiparse 'procedure is to assign all 
as— yet . unahalyzed words to some "phrase. The basic motivation in this 
step is that English is a ' right-branching language, i.e., mo^t of the 
complex const rue tiotfg" in English occur *^n some - right branch of the 
syntactic tree. An example of ,thi^ complex right-branching structure is 
a noun with a relative clause suspended from it as in "John, who came 
home \late last night." The structure for this phrase is 



[John [who came home l^st night] \^ 



one word adjoined to a clause 6d the right .'^ IThile not every- structure 
jLp. English is. right branching, this is. a useful guess for the parser to 
make when it finds no other analysis'. - 

Still more complicated parsing procedures are conceivable, l^at is 
particularly missing from surface parsers is the capabilit^/^fp deefl with 



JU_ Find' the simplex noun phras^es, verb phrases and _„ _ _ 

' prepositional phrases in the sentence. . 
2m Use these phrases, along with unphrased words, to form more 
complex phrases by looking for specified elements and 
then associating other constituents into the phrases* 
a* Some specified elements are searched for from the front 

,of the sentence"; some are searched for from, the end* 
b. Associated constituents may be before or after the 
I ^specif ied element. 
3. Complete the structure by including any unphrased words as 
^ * either (a) or (b). - 
^a. their own phrase, if there are enough words together • 
b. a lefjc sister to some constituent, the created sisters 
' to be dominated by some single node. 



Figure 3. Overview of our surface parser. 

missing arid moved constituents^^* Let us consider /3l simpler-phrase 

structure parser, which ^ is nan-recursive; there are no. embedded 

constituents in its parsed structures. Such a parser, is noJt subject^ to 

this difficulty to the same degi;ee as our full surface parser. since the 

limited* structures available as simple ^^rases are unlilcely to contain 

either raoved^jOr missing * words that make important . structural 

differences. We can compare the results of the surface and 

transforttiktional parsers using the example from our previous di-scussion 

. ■* • 

of stress reduction-, which we repeat below, together with different 

■ . , ' ^, ,- ' f - ' ^ ■ . ' • 

possible p-^jses. Parse (a) would come from the simple-phrase parser, 

(b) would come f rom ,a full surface parser, and, j(c) could result from S 

transformational derivation.* 



^ ^ ^^Missing constituents and discontinuous constituents were a major 
motivation - (historically) for incorporating transf ofmations and/or 
semantics into accounrts of syntactic structure. 



Eager, though I am to win, I would never, cheat. 

) * >'f. [Eager though] [I] [am] [to win] [I] [would never cheat]. 

^ (b) [E^ger [though [I [am [to win]]]]] [I [would never cheat] ]• 

(c) [Eager [though [I- [am [toVin]]]]] [I ' [would never cheat]]. 

y • I ^ 

The focus of attention in this figure is on the different parsings for 

' . . ' • . .. 

"am to win." The simple-phrase parse for those words shows two separate^ 

' * • 11 

phrases. In terms of the prediction of contractions , this parse could 

agree with either of the other two, either incorrectly allowing 

contraction as in (b) because of a tonal assignment upsteppi,ng the word 

''am' to the ^ras^ "to win", or not, as in (c), instead, correctly 

assigning separate^, prominences to the-: word and the »phrase» Parses (b) 

and (c) make conflicting predictions, with , the empirical consequences 

, supporting parse (c)* j, . - 

From a purely practical point of view, we note that the bulk of 

English sentence structures do no't exhibit these deletions and 

movements. The additional complexity due to incorporating derivational 

schemes or semantics into a parser must be balanced against the need for 

4"- ■ ^ ' « ' ■ . • 

* . \ • ■ - ' ■ ' ' ■ . 

correctness in these limited cases* While the surface parser does not 

'always yield the best parse, .it comes very close in a large number' of 

". .. "■ _ . • • . ■ . « . . .■ ' . 

•cases. ■ ' ^ *' \ 

;* " •". ^ . ' ''\ ' ■ r.: 1 

2.2.4.2 Morfe Complicated Parsers 

; • • Two parsers which perform more complicated syntactic analysis than 

our surface parser are those by Kaplan (1976) and Marcus • (197^4^ 1976.) . 

. ; Both .of these attempt to produce structures which would make the correct 



Gf. Selkirk (1972') and Levine (1977) . 
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predictions in the contraction cases cited above , th^ is, exoress 

generalizations about moved and deleted constituents • Neither 'parser 

tries to reproduce the; derivational history of . a sentence as part of a 

parse. " ' * 

/. Kaplan ''s ATN parser 

Kaplan's parser .is an 'augmented transition network. A simple 

c ■ . ^ ■ . • . ■ ■■ ■ • 

transition - network is a graph of connections - (arcs) between possible 

states. ,In a parser each state ^represents- some constituent which is 

recognized wheti 'the parser arrives at that state. The syntactic 

patterns w^iich were 'the- basis of- our parser are represented in sequences^ 

V of arcs and states of the ATN>. Some sequences of states which would 

otherwise appear many times in the parser can be factored into 

subroutines which are separate transition networks. This transition 

V network can be referenced by an arc between two states, as if it were a 

simple word recognition. In this case, the sub-network must give a 

successful recognition* in order for.*the parse to proceed. These sub- 

networks normally represent phrase constructs , like noun phrase, "verb 

phraise, etc. 

-The augmentation of the network comes from allowing the arcs to 
represent ^unlimited) computations. These cornpufati^ns can be used to 
bracket or re-bracket constituents, to assign labels to various 
elements, arid to perform tests besides the simple constituent 
recognition which the states represent. • An impor^f^t augmentation in 
Kaplan's system is the ''hold cell'. - Som^ of the arcs have the 
.possibility of placing a recognized constituent (either simple or 
complex and usually a, noun phrase) into, a designated_^ell, while other 
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Needless to say, this expression of their aims is totally our. 



own. 
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arcs' are enabled to use this cell instead of performing a sequential 
sentetkce constituent recognition. Use of the" hbld cell allows this 
parser to analyze ^ntences with unbounded movement or deletion, which 
. our^- purely surface parser could not, while avoiding reference* to 
derivatlonal^istories. Unbounded leftward movement, which is the main 
example in English, is recognized by placing the (supposed) moved 
consti^tuent into the hold cell and then Whenever a pattern may require 
a cStistituent of that type the hold cell is emptied of its contents. 

Alt important difference between the organization of trhe ATN and. 
that ..of our parser is that the ATN completes its parse of one (the 
current) constituent and then processes the next word pr constituent in 
sequence. It is always building up a unified structural description. 
Our parser will' create many independent constituents and then try to 
combine them into more complex constituents. The difference between 
these two approaches shows up with "garden path" sentences, such as "The^ 
horse raced past the barn fell." The ATN will initially follow a "garden 
path" in mis-analyzing a sentence until it can proceed no further. At 
that ppint it has to back up and recreate at least some of the parse, 

making somp different choices from the structures assigned the first 

f' ■ . . . . » 

■ " ' ' i • r 

time. ^.Iti the sample sentence it will parse "the horse raced past the 
barn" as ^ firi.1 'sentence arid" then . have no' analysis f*or . ""f ell Then, i't 

wiil have to go back and analyze ''raced' past the barn" as a reduced 

*H . * ■ ■ * ■ ' ■ . • 

. -- ^ « ' ■ ■ : r -'-^ ■■ _ ■ - . . . . 

relative clau^;V in Jorc^r to' fit -^fe/l' 'into the , structurie of a 

declarative*. Our parser will always maintain tt(e^ntegrity of the 

simplex structures created iti' the earliest part of the parse, even' if 



. ■ ■ '13 ■ ' ■ ■■■■ "- ' . ■ . ' ■ _ ■ • • • ■ 

- The supposition can be based on the absence, of a syntactic 

pattern utilizing the constituent at that point. 



higher level structures are difficult to ^form from these , constituents. 1^ 
In the above sentence^ assuminj?^ >ast' is marked (or recognized) aX^a 
"l^repositio^^ "'"the ■ horse'\ ~ 

"raced" and "fell" will be^ parsed- in the f first run-through. The parser 

is then free to create any grouping of complex constituents it can. In 

fact" our parser can only create a single structure for. any sentence and 

thus cannot back^up^n case of parsing dif f iculties. 

./Another difference between the parsers' is that an ungraramatical 

""'sente^c^'' will require significantly more parsing than a grammatical 

one does from an ATN but not much more from our parser. This , is because 

the ATN will attempt to recover from possible "garden jgaths" until all 

possibilities for parsing the sentence haye'been exhausted. Our parser 

will simply leave whatever constituents it can find sitting around and 

then give up. The ability to give up quickly is valuable in a setting 

where ungrammatical utterances may be encountered. The concomittant 

drawback is that ' our parser will tend to compound its errors" and. then 

le^ve them sitting whereas the ATN will try harder to get the right 

answer. ^ 

Marcus ^ /"Wait-and-See^ parser 

The ^wait and * see ' parser , (WASP) has similarities both to our 

surface parser^^ and to Kaplan/s ATN. Like our 'parser, WASP depends on 

creating small, simple structures and eventually combining them to form 

the more complex ones, and, like the ATN, it proceeds from the beginning- 

of the sentende to the' end withou|: going backwards and forwards in 

looking for complex constituents. WASP works roughly as follows: 

1) The "parser proceeds from jthe first word of the 
^ sentence and looks for matching patterns (sometimes 

X ^ - .predicting wii^t should be the .next element iti the 
t 'Sentence).. *■ -. . ' ^ . 



""•^Our parser has borrowed somewhat from Marcus • 



*2) If the current word does not seem to form part 
of some constituent (does not natch, a pattern) it is 
stacked on a list of constituents* _ 

3) If it finds a matching syntactic pattern, tr 
words/constituents comprising that .pattern are ' phi 
together and treated as single constituent* ^^S^ ^ 
constituent^ is parsed, it is put on the -ypt^^of 
constituents along with unparsed words* 

4) ^ Forming a new constituent can, matching 
another syntactic pattern, cause already parsed 
constituents to be popped of f the list (last-on first- 
off) in order to b^e incorporated into some new pattern. 

■• ' ' ■ . 'fl 

5) A c^onstituent can have various kinds of 

information associated with it and its sub-cons tituen^g. ^ 
^ In a^nojun phrase, ^the entire phrase might be labeled , 

and the head noun in that saae phrase might be so ^ 
labeled*^ In a sentence, the subject and object (s) might 
be labeled for their 'function as well as for their 
syntactic structure* 

In a sense the UASP employs a different organization- of practically . 

the same' basic notions of patterns and parsing style as are used in our 

surface parser* We can imagine recasting^ our s^^^raL passes across 

constructed constituents into interlocking pattemis, where the formation 

of one constituent allows anothier pattern to activate itself and try to 

find all the necessary components. , Also there is an appealino; sort of' 

ps;ychological plausability to this parser (as well as Ron Kaplan's ATN) 

in the lef t-rJLght direction of attention focus movjement, and the single i 

pass . through^ the" .sentence.. There ■ is no good evidence to support the 

opposite view, that attention scans -back ajid-.forth through' the words 

constituents of the sentence in the way that our parser would have it 

done. ' 



^^It cbuid' receive 'additional labels as part of further phrasing. 
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• ^2.2.4.3 . Semantlcally Guided Par:sers 

The parsers\^e >have discussed here are all syntactically based. 
The patterns ' upon, -^^jKLch '. recognition depend are independent of 




consideratio'n3^''6f t:h[i^^^^^^^> of .the words' or sentences involved, or of 

the structure of vthe ^^^fcbiirsie^"^^^^^ work, on 

parsing 'using world 'knowler'dg^ or ireference to meaning has been done, 

. V^',., ' • ^ ' • • . 
especially b,y Winograd and S.chank- who, despite the differences between 

, t" ' ;\ ■ ■ ■ , " 

their approaches^ 'share the underlying assumption- that' non-structural 

information is important in structural analysis, that syntax is not 

self-contained but requires semantics (and possibly additional sources 

of information) in order to he analyzable.- 

We have not closely IpokeoS. into the 'use of semantically oriented 

parsing .for two reasons^ One reason is simply " the practical one that 

this mor^ complicated jjar^ing requires more resources and has not been 

implemented • in a general enough way- that it could be a candidate foe 

incorporation into this project. The other reason for not pursuinj^ 

semawtic parsing is our belief " that the phonology (intonation) can 

utilize only syntactic structure along with s6me very specific- 

stress/destress information, but without access to. semantic 'information 

to- guide the elaboration of tone groups. '--If the parse > itself depended 

on semantic inforniation any claim in this area would be a vacuous one. 



They are- also independent of -the weather— at the time of the 
parse, but we take it as evident that the reasonableness , of ..usin^ 
meaning .in parsing does not .require the jus ti filiation" that reference to 
the weather does. , . . . ^ " . 

'17''' . ■ ' 

Samples of different approaches along . these lines are contained 

in Schank and Colby (1973). / . 



2*3 Outside Contacts 

During the past year^ we have had numerous contacts with Professor 
Jon Allen of MIT. He has provided us with updated versions of his text- 
to-phoneme programs. His programs are written in the language "BCPL". 
The TENEX system in use at our site also has a BCPL compiler but the two 
compilers have vast differences both syntactic and semantic. We were 
able to convert most of the text-to-phoneme programs to TENEX BCPL and 
now have them operational* We currently use his text-to-phonerae program 
as one part of our word analysis routine for dictionary storage of a 
sound. When a word is recorded, there is always some silence before and 
after it in the recording. For good concatenation this silence must be 
removed. By 'using the MIT program to provide a phonetic transcc^iption 
of the word> our analysis program is able to more accurately deterrnine 
the boundary' between silence and word in our recordings. 

We have also exchanged information, through personal contacts and 
-regular correspondence, about how our different intonation algorithms 
work, including plots of parameter waveforms both| before and after, 
application of our algorithms and representative samples ^ of the 
sentences our curriculums use. We . have also provided him with some 
programs of; a general / nature and our lent our expertise on the 

suitability of certain peripherals for the recently acquired computer 

• - / . ■ ' ■ 

system, which is fairly similar to and largely compatibSfe with our 

systefa. -r ' 



3 • Complex Teachlnfc Prot^raws with Audio 

The Institute has developed three large scale, college level, 
"laathematlcally "oriented ~CAI " courses: " Eleraentary^^ 
Theory, and Proof Theory. All three courses are used by Stanford 
Universi*ty as regular parts of the undergraduate curriculum. Students 
receive three to five units of college credit for these courses, and use 
them as prerequisites for other, non-CAI courses at Stanford. The 
courses also serve as an environment foj; the study of learning and 
teaching methods. This section describes the work done in the* past year 
to further develop and extend these courses, and the results of the 
experiments performed during the past year on various aspects of the 
courses. ' . * 

3. I 'introductory Logic Course 

■ ■ ' ■ ■ . ■ ' 

3. 1 . I Audio/Display Interaction in the Lo^ic Course 
The first portion of the logic course to be rewritten in VOCAL was 
that which dealt with translation from predicate' logic to English and 

vice versa, . since it was .initially thouglit that the addition of audio to 

...... ^ - .' ' ' ^ 

the logic course would have the . most impact on such paraphrasing 
exercises. While the prosodic emphasis and the informal Cixplanations 
available in audio mod'fe did aid in the understanding of these important^ 
semantic concepts, it turned' out that audio made a greater impact on a 
different area of the course. 

When the remainder of the logic course was rewritten^ we found that 
the primary advantage of an audio /display presentation over simple 
display was in the demonstration of processes, such/as an/ example of the 



use of a new inference rule. With an audio-supp^mented presentation, 
^ only the material which would actually be ^yped during, a ^derivation need 
appear on the display, thus preventing a confusion of explanation -with 
object. For example, rather than using artificial devices " such as 
bracketing to indicate text^.,whlch is presumed to be typed by a student, 

haveVhe program speak "you type the ■ line, number" 
as a number is being typed, and speak "the computer will then print the' 
resulting formula" as .the formula is being printed. Thus, the display 
invariably looks clearer and less cluttered when . comments are reserved 
for, the- audio. - 

The spoken text also allows one to add , a dimension of timing to the 
process being demons t rat ed,^so that its steps can be done one at a time 
as they are explained orally. The author can coordinate a display 
action with the time when a student hears a spoken comment (even though 
the student controls the speech rate), but there is no way to know when 
a student finishes reading a written comment, except "by use of the HOLD 
opcode, and overly frequent^* HOLD 's become annoying. T4ius, in the ' 
display-only .versions, we had to type out a large section of the example 
all at once, parallel with a large 'section of explanation, and leave the 
student to jump back and forth from one area of the screen 'to another, as 
he reads. The loss of clarity is very, dramatic, but" it is impossible to ., 
demonstrate' this adequately in a report which itself must be committed 
entirely to the written page. 

We will attempt to convey the effect somewhat by illustrating the- 
successive display contents, wlgh ■ spoken • comments below. ' ' In the 
following example, adapted from an actual lesson, double divider lines 
mark places where the student can have the spoken text repeated 



or *'hoLd" the "presentation until he is done' examining the display, and 
single divider lines (— — ) -Indicate places where something * chanp.es on 
>the display. Boldface type in the display content represents 
brightening (display^ In double intensity), and underlined type in the 
spoken text marks words which have been tagged for extra.^prosodic 
empliasis. The actual .VOCAL code which produces this lesson follows tl;ie 
Illustration. The VOCAL author manual, which was written this summerr 
contains further discussion of audio/display interaction and several 
more samples of lesson code* . ^ . 

... ; ' " . ■ . " ^ 

- . SIMULATION OF THE AUDIO VERSION: 



P 
P 

■k 



(.1 ) 2 + Y 
(2) X = 2 



=8+X & X-3+1-X 



SPOKEN; 



P 
P 
*1 

SPOKEN: 



P • 
P 

*1,2 
SPOKEN: 



"Our two new rules work a lot like Rules R 0 and R Q except 
that instead of replacing equivalent sentenceis , • they- rep la„ce 
equal terms . For all four of these rules," " • 



(1) . 2 + Y- 

(2) X = 2 



= 8 + X ' & X = 3 + 1 # X 



"the first number indicates the line in which the replacement 
is to take place, " ' . - 



(1) 2+Y^8'+X & X = 7 + 1 - X. 

(2) X =2. 



"and the' second number indicates t^e line which justifies the 
replacements For RE_and..RER, this line must be an equation." 
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p 

_*1,2_ 



(1) 2 + Y - 8 + X. & X - 3 + 1 - X 

(2) X » 2 



SPOiCEN: ■ "To- replace occurrences of the left term with the one- on the\ 



right," 



P - 
p. 

: *1, 2RE 

SPOKEN: 



(1) 2 + Y = 8 + X &, X.» 3 + 1 - X 

(2) ". X = 2 ' 1 



"you use Rule RE. If you don't put any occurrence numbers 
after the rule," 



p • 

*1;"=2.RE 



(1) . 2 + Y . 8 + X & X = 3 + 1 - X 

(2) X = 2 ' 

(3) 2 + Y 8 + 2 & 2 •= 3 + 1 '- 2 



SPOKEN:' "all of- the occurrences will be replaced.' 



P 

* • 



CL) 2 + Y = 8 + X & X = 3+ 1 - X 

(2) X = 2 . - , - ■ . 



SPOKEN: "If y6u don't want to replace all the occurrences, then list ; 
^ . ■ the numbers of the ones you. do want to replace after the: 

name of the inile. "ffBr example, if we only wanted to replace" 



P- 

*. 



(1) 2 + Y = 8 + X ' & - X = 3 + 1 - X 

(2) X = 2 . 



SPOKEN: ~ "-the third occurrenCjC of X here% 



P 



J- 



(1) 2 + Y = 8 +. X & X ,= 3- + X 

(2) X =• 2 ' 



■*1,2RE • , ■ . 

SPOKEN: "then, we would type 1,. comma, 2, R E," 



P c . 

p 

*1,2RE3 



(1) . 2 + Y = 8 +.X & X 

(2) X = 2^^ 



= 3 + 1 - X 



. SPOkEN;: "and finally . a -three , " ; 
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P (1) 2 + Y = 8 + X &. X, = 3 + 1 - X 

P (2) X = 2 

*1,2RE3 (3) 2 + Y = 8 + X & X = 3 + 1-2 



SPOKEN: 



"and hit escape, of course. " 



P 

•P 

*1,2RE3 
* ' 

SPOKEN: 



./ (1> -2 + Y = 8 + X & x:= 3 + 1 - X 
: (2fec = 2 ° • . 
i^Ci + Y = 8 + X S X = 3 + 1 - 2 • ' 



"If we wanted to replace the occurrence of 2 in line 1 with 
an .X,"' ■ - - , = , 



P 

*1,2RE3 
*1,2EER 

SPOKEN: 



(1) 2 + Y>>8 + X & X = 3 + 1 - X 

: (2) X - 

' (3) 2 + Y^^= 8 + X & X = 3 + 1 - 2 



. "we would use the Replace ^Equals (Right) Rule, which i,s - 
viated R E R,' instead of Rule R E. Since there is only one 
occurrence of 2 in 'this line," ■ . 





(1) 


2 


+ Y 


■= 8 + X 


& 


X 


= 3 


+ 


1 


- X ■ 




P' . - 


(2) 


X 


= 2 


















*1,2RE3 


(3) 


2 


+ Y 


= 8 + X 


& 


X 


= 3 


+. 


1 


-2 




*li2RER 


(4) 


X 


+ Y 


= 8 + X 


& 


X 


= 3 


+ 


1 


- X 





SPOKEN: 



"ao occurrence ^ojumber is necessary." 









==^=r==«-= =-END=OF-EXERCISE 










THE VOCAL CODE 


FOR THE, ABOVE EXERCISE: 


















[EXERCISE 3 


"RE i 


i RER: "Tleplace Equals' and ''Replace Equals 


(Right)'" 




[AUDIO 












(TEM2 " 












P ' 


(1) 


2 + Y = 8 + X 


& X = 3 + 1 - X 








XXX 


. A. B 


■ c d'/ 






■ p ; • 


(2) 


X = 2 












z w 




• ^ %3_ 




*li2RE / 


(3) 


. 2 + Y 8 + 2 


& 2 = 3 + 1 - 2 




aXcYRR. 


PPPPPPPPPPPPPPPP E 


q . F\rrrrrrrrr G . 






*1,2RER 


(4) 


X + Y = 8 + X 


& X = 3 + 1 - X • - 


. 7A_ 




buuuvvv 


ttt 


I uuuuuuuuuuuuuuuuuuuuuuuuuuuuu 






*1»2RE3 


(3) 


2 + Y = 8 + X 


& X = 3 + 1 - 2 ' 






eS'SSSST 


ssssssssssssssssssssssssssssssssss H 






II 
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(HOLD (S (T 12 a) - • „ 

"Our- two new rules work a lot like Rules R Q arid-R Q R, ' 
"except that instead of replacing S3 equivalent $2 sentences 
"they replace $2 equal $1 teriais." .::..^-f. 
"For all four of these rules," - . * . : v 

(B X x) V \ ^ 

"the first number" indicates the line/ in which ^ttte 
"replacement is to take place," , 
((U X x) (T c) (B Y y)) - . ^ 

"and the second nuciber indicates the line which" 
"justifies the replacement*" - - ^ 

"For R E and R E R, this line must be an equatioti.") - 

(S ((U Y y) (B z B C D)) 

"To replace occurredces of the left term with" - 

(BR) * , ; 

* "the one on the right, you use Rule R E." 
" (W 250), " :^ * ^ „ 

"If you don't put any occurrence numbers after the rule^ 
((U R z) (T p) (BE) (T q) (B F) (T r) (B G)) 
"$2 all of the occurrences will be replaced.") 



) 

(HOLD 



) 

(HOLD 



(S 



to replace 




((U B C D) :(0E 3 13) (T e)) 

"If you don't want Jto replace all the occurrences,.' 
"then list the numbers 'of the one§^you $2 do want t 
"after the name of ^ the rlile." 
(W 250) - ' 7 

"For example, if we only wanted to. re{ 
(B D) ^ . 

"the third occurrence of X here,' 
(T.S) : 

"then we would type 1, comma, 2, 
■ (? T) 

. "and finally a three," 
(W 250:f ' 

"and hit escape, of course.") 
(W 1000) m T) (T s) (B H) (r b) 
) (COMMENT end of- hold") „ . 

(HOLD - (S ((U'D H) -(B A)) ;: ^ ■ T. ' 

, "if we wanted to replace -the occurrence of 2'' ■ ' ^ - : '* 

' ■ ;(B w), . - : ^' ■ ^ . ■ 

"in -line 1. with an X," . . . . : . . -. r : : 

; ■ (TT u) (B;V)y V V . „ 

Ve would use the Replace Equals ''(Right), Rule, - . 

"which is abbreviated "^R E R', instead of Rule Rr E. 

"Since there is only $2 one' occurrence of $3 2 in tliis line, 

. (tf/v) . - . ■ : \, ' • 

'"no occurrence number is necessary.") 
(W 1000) (U w) (T t) (B I) (T u) 
) (COMMENT "end .of hold") " 
(S (U A I)' 

"The next exercise will give you some ^practice 
"with these two powerful rules of inference.")-. 
)]1 (COMMENT "end of Exercise") 



■ >.3vl.2 ^ Student preference In -^udio-nonatidio .Choice Situation ■ 

' During the winter^nd spring quarters of the 1976-77 academic year, 

166 students were enrolled in .the logic course. .Data was collected on 

' . . " ' ' ' , ■ . ' . - ' 

donnecC time- in exercises, audio choice at login, and calls ; to Browse^ 

. tapde. '-Students were divided into two groups, each of which was exposed 

to audio and nonaudio versions of an initial segment of the course. In 

the winter quarter half of the students were exposed to audio in the 

first three lessons and nonaudio in the next- three; the othet half had 

nonaudio followed by audio. Beginning With lesson 7 and ^continuing 

through lesson 18, the students were free to* choose, at each. login, 

either audio or nonaudio versions of ^he course. During the spring 

quarter i . the , forced switching initial segment . was reduced ^o two 

. lessons, and data was collected through'^^les.son/ 20. In addition to this 

■' . .. . ■• ■ . , ■ . * • ■ . ■ - • _ • - ■ - •*» 

expeisimenti, the springs groups were further divided . into tWa^groups for 
^ each original group. Half of each of the original groups were flagged 
for precompiled synthetic prosody as opposed ^.to "llong sounds' (see 
(Hinckley, et./'alV,., 1977) , so th^at if .they chose'audio at login during 
or after y lesson .-21,. they . would hear. ■ the synthetic-^prdsody.. 
Quest tonaires 'were v used 'dUjiring the spring .Quarter to provide some 
background o^ the students ■''. view of the course, the audio component, and 
the reasons for their -Choices. ^ 
The data collected from these experiments is still in the process 
of being analyzed, mainly with view toward generating a stochastic model 
of the "students' preference in terms of the choice paths. Forthcoming 
articles and technical reports will provide detailed accounts of* these 
experiments and their analyses. \ ... 

. ; In .'addition to the above experiments,; a study was. conducted on 
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rcises require 



students' behavior in interpretation exercises. These e> 
the Student to generate a counterexample in* the domain of integer 
arithmetic for. an invalid argumep.t, and #theh prove that their 



More complex* 



. ^interpretation of the argument is indeed a counterexample 

exercises of this^ type require the student to first decide whether a 

given argument is valid or invalid, then prove the argument or generate 
' -.f ".- 
> . a counterexample. Another use of this type of exercise is in > showing 

the consistency of a set of_ statements. . 7/ 

Data from the interpretation exercises, including use of the ^hint' 

feature, were collected in the winter and spring quarters of the 1976-77 

academic ye^r'» The :data were collected- f6r two purposes: first,Vtq 

predictf the difficulty, a student would have on a particular exercise 

from, the structure of the exercise; second, <to find a stochastic model 



that would describe the student behavior on the interpretation exercises 
where the student has to 'decide whether an argument^ is vallS or invalid. 



'-^ ' ^ ' . ' .' V/v ■ ^ ■ ' ■■. ■■■ - ■ V.--: ■ ■ '. .. \ ■ • 

Data-^analysis .is now being performed and- the Results of the 

investigation are expected to be published s^TOmfer 1978;i 

(iissertation by Inge B. Larsen. ' ^ , • 

■ ■ ■ .■ ■■■..::>:.■• ■ .; . j: -'^'- '^-.-'- - 

3.2 Set Theory Gburse ' : . ■ • 

3»2*1 Audio Introduction to EXCHECKr Based Courses 
, • The goals of the OVERVIEW program* mentioned in the proposal have 
been expanded to - include a historical model of each student's style of 
•probf buildihg^ as well as the model of • the currient object dialog. As a 
.consequent, this program is still in the development stage. Rather than 
wait for its completion before writing an audio introduction to EXCHECK, 
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though^ we have utilized a HELP system, which was originally written for 
the'^logic course, to perfornl this function* 

:^ The HELP - sys.tem is completely 'implemented. set 'ofX^hefp 

modules', similar to the explanatory exercises of the logic, course, are 

^"a^ ■ " ' ■ ' ■ ■ ■ • • • •' •■• ' ■■ • . ■■ • ■ 

written, in VCXJAL* * They /can include derivations, and other - types of: 

"•"questions, bp t such-Vexercises are for assistance, only.. They ' are" not 

"scoredV-, and the student may skip then if he desires. The system is 

' ' . • , • - ■ ■ -. , 

meant to contain- a help module for each topic which stlidents may need 
tutorial— style assistance on. It* can be expanded by the teaching 
assistants as they encouriter 'student problems. The course authors also 
maintain a[ "graph" of how the various modules- in tfie. course HELP system 
relate to each other.' . V 

Unlike the Intiended OVERVIEW program, HELP does not itself keep 
track of how the student, is performing in the course, nor does it 



interrupt and ^volu^teer- 



information^ to a student who is having 



trouble. Rather, the system'' is called by ^the course ' driver when, a 

student types ^HELP ', and then the student is given : a sefcLes of 'topic 

choices intended to narrow down hi^ particular 4rea of difficulty or 

interests He may then-ask for one of these topics,^ or-any other topic 

• ' - ■ ■ ' ■ ■ . * 

which he knows the Wname of , and the req^^ested audio heipmodule will.be 

-presented. A ^ further . list of . related chod>ces (specif ie<i by the 

Aforementioned graph X is thfen presented,, and the procedure repeats until. 

the student asks to return to the ouLer course. 

The HELP program is passed two, arguments by the main driver: the 

lesson^ number which the stiyient is currently at, and a list of ^''opics . 

which may need special emphasis. Each lessori has associated with it a 

subset of the helpmodules which aire particularly relevant to the 



inaterial presented xh *the lesson; The special eitiphasis* list is ^sxxally" 
null, but if . the student Has recently been given an error message, it 
will be set to include any relevant topics , (e.g. to. the modules on 
■ syntax when a student has entered a formula which will not parse, or to 
the modules . on -quantifier restrictions wlien a student, has attempted , an 
•invalid use of a quantifier rule). The lesson emphasis' and error mode 
emphasis modules are then added to the initial 'list of topic choices. 
Thus, for the HELP -system to become more, responsive tq. the student's 
particular needs, no modification of the HELP program itself is needed; 
rather, the ma^ driver, with the help of OVERVIEW, need only become 
more sophisticated in, its choice of argu^nents to pass to HELP. 

Since EXCHECK is used for se^i^ral courses (currently ^ set theory, ' 
proof theory, and the A-grade sequeoce in ' probability theory o£ the 
logic .course), some students who. begin one. of ^'these\;ourses. will have 
already encountered EXCHECK in another course. Others may be^familiar 
with the general operation of our computer assisted instruction system 
through courses, such as • "Introduction to Logic", which do not use 
EXCHECKw The HELP system, is thus an especially appropriate vehicle, for; 
the introductory ^equence in the use -of EXCHECK and "the rest of the 
instruction . system. Students can , ask to view just as ra^uch of the 
material as is new or useful to. them. ^ y 

Therefore, ^11 the introdtictbry material not specific to Set Theory 
was pufc^into audio help nodules J .The following list, taken from a 
recording of the topic lists output in a test of the HELP system, is 
representative of the raore than 150 help topics currently included in 
the EXCHECK. sequelice. Most have associated tutorial output, . but some 
topics (like ADMIN, SYSTEM, and QUANTIFIERS)' are used only to guide the . 



interrogation which leads to selection of an- appropriate module* (The 



bracketed letters indicate the minimum string which a student must type 
for the system to recognize which topic name is intended*). 



ADMIN 
SYSTEM 
AUDIO : 

REPEAT'-.; 
BROWSE • 
HIKT-. : ' 
SPEED 

BACKSPACING 
ZAP 

EXERCISES . 

GRIPE 
, NEWS 
. EXCHECK 

WORKING 

SORTS 

META-PROOFS 

REPLACE 

REP-SUMMARY 

REP-EGi 

REP-EG2 

RER 

VERIFY 

VER-LIMITS 

UNAVAILABLE 

REMOVE 

ABBREVIATE 

OWN~F0RMULA^ 

QED : 

SCOPE 

BOUND* 

QUA^T-RULES. 

QUANT-RESTR 

AMB-NAME 



[AD] ; V, Administrative matters 

[SY] , Communicating with the computer system 

lAU] ; Problems with . the audio -system 

[REPE] ' Repeating the most recent material (TA) 

[BR] Using Browse Mode (TB) 

[HI] Getting hints in derivations or questions- (fH) 
[SP]- . Controlling the speech rate (TS> ' 

[BA] Erasing mistypes with TW, TX, & tlie DEL key 

[Z] Logging out or leaving a subsystem (TZ) 

[EXE] *. The various types of exercises^'in the course 

. [GRI]^- How to send a complaint or suggestion 

[NEW] How to. ask for news on^ the. course 

[EXC]. The use of the proof checker ' . . 

[WO] - The use of working premises - . 

[SORTS]" The sorts of variables arid terras 

[META-P] How to prove J^heorem schemata . ^ 

[REPL], The Replace'rule 

[REP-S] ^ Summary of the operation of REPLACE 

[REP-EGl] A basic example of the use. of REPLACE- 

[REP-EG2]- An example using more/features of REPLACE . 

[RER]^ The Replace' Equals (Right) rule l''^'' , A 

•[VERIJ The Verify rule 

•[VER-L] Limitations on the operation of VERIFY 

[UNA] Rules from Logic 57 which don't work here / 

[REM] Reinioving proof lines from tl^e display region 

[ABB] Defining and using your own abbreviations 

"'[W]. Proving your own formulas with SETDERIVE . 

[QE] . IThat the QED command does for. you 

[SCO] The scope-of a quantifier ' 
[BOJ. V ■ '. The definition of bound variables 

[QUANT-RU] The Ilules which manipulate quantifiers 
[QUANT -RE] ; Restrictions oA the quantifier; rules 

[AM] Using vaViables as ambiguous names 



In addition, a short sequence of audio exercises on the language of 
set theory accepted by EXCHECK was written, to be presented as the first, 
•lesson of the Set Theory course. iSince the languages differ in some 
ways from course to. course, this material will not cause repetition' 
problems when presented to'* all set theory students, as would; the 
material on inference rules and general system use. One of these first 
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y^^rclses explains how ' to e^er the HELP . system, and suggests usirn? it 
• to viev all the int^roductory material with which the student is not 
already familiar. 

Formerly; the only place this introductory material existed was in ^ 
'.a course manual* The on-line course contai;aed frequent injunctions to 
read a given section of the manual before proceediag;,. to the next proof • 
Now that all this material is included in a much more instructive, 
interactive form in the HELP system, the manual can be purged of its 
tutorial-style sections, '-^nd made into a smaller , and cleaner reference 
nianual to be kept at ones side during a session at the terminal, rather 
than read in preparation for such a session* 

3.2^2 Introduction of Audio Capability in EXCHECK 
IThen comp-utex-synthesized speech first became available for use in 
GAI, a primary concern of the DISSS research staff was' its effective 
utilization by course designers, curriculum authors and students. The 
Stanford logics course provided the appropriate environment for 
investigating these problems since it ^ had the most well— developed 
curriculum (written and extended over a period of many . years by a 
■ -diverse group of authors) and a consistently large student enrollment 

(120 students enrolled .in the logic course during the 1977 Spring 

- J ■ ■ ■ ■ ' . ■ . -r 

quarter.) Therefore the initial implementation of the programs to. 

support audio and, coordinated visual displays were specif ica'lly. designed; 

for the logic course. • 

During. the past year the capability for computer-synthesized speech 

has been added to programs associated with the more matti^eraatically— , 

sophisticated EXCHECK system. This' addition represents a major step 

■ ■ .- ■ ' ■ ' ' ' ^' ki . ■ ^ : ' „ , .; j .■ ' 



.away from the experimental usage of audio in CAI courses, ancf toward the 
use of audio as a standard system component in CAI* - 

The Itris^^ step - in extending audio capability was to. to allow 
lessons for : all the Stanford CAI courses to be written with aa audio 
component. / The.; VOCAL lesson compiler /interpreter, developed for the 
Stanford CAI logic course, was extended to allow audio lessons to be- 
compiled and- tested for any - EXCHECK-b^sed course. Essentially this 
extension involved merely allowing the usfe of arbitrary parsers* 
However/, to use this program' efficiently under the TENEX: operating 
system, the program had to be restructured at two levels: the source 
files' had a nev compilation structure- imposed on them, and the runtimel^ 
program had a new "fork*' structure added* The resulting runtime program 
now has a main fork (process) which contains entirely shared code, and 
an audio fork which is also shared among all users* The only non-shared 
code consists in the parser fork, which may differ per course* /The fork 
structure was not critical when* computer-synthesized speech was used 
only, by writers of logic lessons since they all ..used the same 
(sharable) program; however, with essentially the same code in use for 
different courses, proper utilization of the TESEX operating system 
requires complete sharability of ;programs* 

Additional problems, in moving from a purely experimental ^design to 
a design, admitting widespread application were attacked. Most important 
after proper system utilization was the problem of producing code which 
ran .more efficiently than its experimental forebearers* Efficiency in 
String-handling and list processing made strong demands on the exitended 
TENEX-SAIL compiler, especially o.n the SAILISP .ext^n^on* ^ Design 
problems in SAIL and SAILISP were found which; necessitated a rewrite, of 



the latter* In restructuring the compilation of source files, toany 
steps were taken to make the code more efficient ^efficiency was- 
increased in- .both string and S'-expression handLin<^) • 

In ; addition, the new cor^ilation ^structure; provides a greater 
degree of "modularityy which makes it relatively simple to implement 
local program changes. The initial logic program with audio capability 
required about 90 minutes of runtime to compile and load* (In addition 
it required extensive • and inefficient' use of - di$k space.) Minor program 
changes often required the full 90-minute recqmpilation sequence* Such 
an expensive compilation sequence would not allow us to extend, audio 
capability to provide- students with analyses of their -individual proofs. 
With many" individuals now ^working on. program modules for different 
courses it is essential to allow^J^Tocal" program modifications > to be 
incorporated without lengthy recompilations of the entire system.' -The 
restructuring of the course drivers (which handles the audio and display 
components) will allow such local modifications to be made with only ^ 
small eGcpenditur^ of computer res.ources.^ .... * 

An analogous but even more extensive restructuring of the EXCHECBC 
. system to incorporate* audio capability in. its driver program is now in 
process and is. expected to be* completed in September. The development 
of a dynamic' system for -proof explication using audio has been 
temporarily discontinued, pending comnletion of the introduction of 
audio capabilities in EXCHECK, and improyed real time ^eneratiori of 
audio messages with synthetic prosody. \ 

. \ 3* 2,3 , The EXCHECK Proof Checker * • ' ' y v 

The . proof checker used in the EXCHECK system is a general-purpose 



proof checker for manysorted axiomatic theories* A description 'of /the 
proof -checker and previous refinements— -to.^— it ._^m ay T>e- _f ound in. 
(Smithv; Graves, Blaine and Marinovy 1975) , -{Smith and Suppes, 1976) , and 
. (Smith^ahd Blaine, 1976)1 The ' hasic jJesi^n philosophy of the checker is 
to accept proofs presented' in the , style of , standaard mathematical 
practice. That is, just as the goal of a natural language system is to 

C 1 . 

understand language as it is actually used, . the goal of the EXCHECK 
system is. to understand and check proofs as they are actually presented.. 
We -are as yet a considerable "distance f rom that goal but* irf the last,, 
year progress has been made in making more natural the basic commands -of 
the proof language used by the students to, express their proofs. 

3.2.3.1 'Decision Procedures - 

It is common in standard mathematical practice simply- to state, as 
obvious elementary mathematical results r£::her i-^an to construct 

Sexpiicit derivations' of those results fr.om axioms and theorems. The ^ame 
freedom can be provided in proof checkers^or those parts .of elementary 
mathematics or logic for which there is a feasible decision me tl:io<f. One 
such area is quantifier-free 'boolean algebra ' or, equivalently,- 
quantifier-free set algebra. The EXCHECK system contains an inference 

•'rule BOOLE based on the decision, procedure f or . quaritif ier.-f ree set 
algebra, an inference rule TAUTOLOGY .based on a truth table decision 
procedure, inference rules VERIFY and IMPLIES based on a resolution 
theorem prover. The TAUTOLOGY and BOOLE ' rules were described and 
illustrated in prior reports. In this last year a new infer^ce rule 
TEQ was added for use- itl^ Inferences involving oni^ tautolo^^y and 

-identity. " • • ■ ■.■ ■ \ ' , 



The TEQ rule will accept most inferences 'that can be '^obtained by 



repeated use of the sentential rule and identity rulesw In particular. 



w It/* pa 



handles «the congruence properties of identity as* can be s^n from the 
exanple below. Also, see the example f or the IIEPLACE rfele for another 

*WP'- (.1) = B 

*WP (2)' *Pow(A) = pow(C) ^ \ y ^ , 

^r,2teq$ (3) *Card(pow(C) ) = card(powCB) )$ 
Hill you wish to specify? (No) 
Using *g$0 



Decision procedures, such as those used ia the BOOLE . and TAUTOLOGY 
Wles, if they are to be used in programs .'for informal mathematics, have 

to be HEeasibl© arid should decide a perspicuous class of statements* The 

) ^ ■ . - " " - • - ' *\ 

.procedures used Aitr^BOOLE, TAUTOLOGY, and TEQ satisfy these requirements 

bu£ the resolution procedure used' in VERIFY and IMPLIES does riot satisfy 

t-he- requirement pf perspicuity. In the last year the VERIFY and IMPTjIES^ 

rules have be erf augmented with natural deduction heuristics to make them 

correspond more closely to what , users find obvious. 



_3.2.3,2 Sorts 



^ Elementary mathematic^ and logical facts are another related.,kirM 
of detail that mu^st o^ftek be handled explicitly in proof checkine: 
p-rograns- while they are almost never handled explicitly- in informal 

• ^ - ■ , :^ . / ■ / • 

proofs. Sorts of complex terms are a good example of this kind of 



detail. If a program 



is to accept natural proofs it will ha^e to 



implicitly handle suAi details as the sorts of complex terms. Sorts 
might be unusual, however! in . that they can- .rather, neatly be implicitly 



handled — taken to J>^-^ part of the implicit context as in standard 



practice. Most of the procedures for the implicit handling ' of Sorts 
'were rewritten this past year, with a considerable gain in efficiency. 

• ' In the set. theory course there are curre.atly five' basic sorts: 
geaeral, set, .function, ordinal, and; cardinal. ^ Associated vwlth each 
sort is a group of variables that ran^:e over that sort* In the current 
set theory course, ""A' -and range over sets, while "^a*" and ^b" rans?e 
over ordinals. Hence, q^ie statement that for every set . thete^^ts^ an 
ordinal equipoXlent to it could be expressed: for every A there is a b 
such that b-- is .equipollent to A* The sorts aire closed under untori, 

intersection, and re^lative difference and form a set algel^ra; Hence,, 

J ■ ' ' ' ' ..... 

the relation of inclusion between sor^ts is ••decidable. 

Complex terms also have sbrtis in ^ that they denote objects that are 

sets, or ordinals, or functions, or the like. In bur version of set 

theory, ordinals are . -sets. In^fact, aif- ordinal- is the set of all 

smaller ordinals. *lt follows that the ^^ntersectdon of two ordinals is 

an ordinal — the smaller ordinal. However, riot ^^1 sets are ordinals and 

the intersection of two sets might be a set that is not an ordinal. 

Hence, the object denoted by a . compound term formed, using ^"^^ might, for, 

*'e:t^mple, .-be ordinal or it night be a* set that is not an ordiTial. The 

EXCHECK program must determine a sort for .complex terms before it can 

substitute them for sortfed variables. * Rather jithan have the user 

explicitly establish t^^^ort of a complex term,, the program tries to 

compute the sort on tlie^ basis of information it has available to it. The. 

.curriculum authors supply EXCHECK with basic information about the sorts 

of variables and, for earch function symbol, informatior about how the 

sort of a compound term formed using, that operator is related to the 

sorts of its sub terms. For example, p^t of , the information for "T' is 
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. ttiat.'if-^l^th subf?^rms are ordinal, -then the compound term is an ordinal; 
otherwise, if .both are sets then the compound" is a set* 

Using, the information available to it., the EXCEECK ' program will 
compute \d sorZ far any compound term. However, occasionally the sort^ 
coii5>uted is insti£ficient to permit the. desired inference. In such cases 
the user., is required to supply information justifying assigning a more 
restrictive sort , to the. compound term. Once this is done, the 
inXbrmation about the, new sort is saved so that: the student need not 
repeat the p3::ocess each time the temn is used. Information about the new 
sort is store^d on"' one of two lists depending upon whether or not 
jextratheoretical assumptions are required tb establish the new sort. If 
no extraitheoretical assumptions are required, the result about, the new 
sort "is a theorem an^l' it- is made Available as a standard^ part . of the 
implicit sort machinery If extraGheoretical assumptions . are. required, 

the result is only made available in the context of those assumptions. 

- * ' ■ '■' . ." * ' . ■ , . ■ 

■ , ' ■■ "- . v- • . ■■ • 

. 3. 2w 3-3 Schemata • . ■■ . ^ ' 

The instantiation .of' axiom and theotem schemata is . an area where 

■ . " - " ■ ■ ^ ■ . ■ ^ ■ 

some e'ffo.rt must be made to provide /"routines that do not involve the 

user in logical details. The procedures involved in '^he instantiation of 

' . ■ . " " * ■ t 

schemata were completely redesigned and rewritten in the last -year. They 
were extended, to - automatically handle almost^^larfLl of the lo^'gical detail 
involved when used explicitly by the student and to automatically do all 
of the work when used in conjunction with IMPLIES. 

' ■ . ■ 

*In standard practice one simply says o;t writes down the appronriate 

'. t * »*■■«■■* 

instanc^^ Using proof checking programs it in'^lves , less work to specify 

the instance and. let the" proj;ram generate it'. Further, the -same basic 



td compute! what Instance to use in an 
Involving a^ 




x*" would replace .^e 
parameter place and 
hic^ marks the ^sec^nd 



ii) . Subs tituti'h^ the formula which; results /rem i) 
for *FM(x)/ in the schema • 



;V. . In both steps variables will, be rebound to avoid capture or clashes of 
bound variables as needed- 

So far the handling of schemata is quite straight forward; however 
the case where the sort of the parameter in the schema differs from the 

sort of the parameter in the desired instance requires more care* 

■ ■ ■ ' ■ \ ■ ' 

TJie approach we have taken is to modify the" algorithm to tibte the 

sort of the variable being used to indicate the parameter places. If 

■ - ^. • . . .-^ •■ • . . " / ■ • 

this differs cfrom the sort of the corresponding parameter in the schema 
then it Is regarded as an instruction to generate the instance where the 
parameter is of the 'new sort. To do . this, the program substitutes a 
formula that 'is made up by first replacing the variable by the parameter 
in the ^chema and then forming the 'exjunction with, , the", assertion that 
the parameter is o-f the new. sort. The program then rewrites the- r;esult " 
in the new sort where possible. (la fact the .'code ia more sonhisticated,' 
accomplishing everything in one pass.) The following two examples shoald 
. 'make this clean;. In the second example,, the bound parameter cannot be 



rewritten in the new sort. ^ 



1. 
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\*th$EOREM (Number or Name) *0> IS \ 
Schema:' ' -•:.*..;. "'"^ • 
If (Ei x)m(x) thert (E x)FM(x) 

Replace for * A pow(B)$ ^ 

Which variable indicates^the parameter pteces^ 
Th. 0.1 Instance: A = pow(B) for FM , - 
V * (i) If (EI A)A » pow(B) then (E A)A = ppw(B) 



What has happened here is that the \ program has implicitly 

substituted ^set(x) & x = pow(B) ' and rewritten the result replacing^.. 
"(Ei x)(set(x) & X = pow(B))' by "(EI A)(A = pow(B))' etc* ' 

*ax$I OM (Number or Name) *se£$ARAT ION 
Schema: ^ ^ 
(E C)(A x)(x in C <-> x in.B & FM(x)) 

Replace for. FM * A is a subset of D$ 

Which' variable indicates the parameter places? *A$ 

(E C)(A x)(x_ in C <-> x in B & setCx) ^ x sub D) . 

Do you want to specify for B *ri$ 

Av. :jE1:'/J?jITI0N- instance: A sub D for FM 

(1) ^A B)(E C)(A x)(x in C <-> X in B & set(x) & x sub D) 

In thi^ example the • program implicitly substitute^' 
'*set(x) & X sub but could not rewrite " the bound narameter "x' as a 
set variable because the rewritten f ormula is* not a conseque^nce of th^ 
result of the i:aplicit substitution and the sor^axioms. 

In summary then the modified alj^orithm accomplishes what is 
desired: it allows the user to specify the instance he wishes without 
requiring him to confront distractin g l ogical details The procedure is 
best possible in the sense that it is complete with rq3pect to the sort 
axioms:' it - will ' allow every instance that, can be* obtained by 
instantiation and. rewriting sort's^ — every instance that is, so to speak,* 
a sort consequence of the. schema. .^n — ; . 



3* 2. 3. 4 Lee Rule. 

During the year a new inference , rule LET was added to permit 

the introduction of a object with certain properties provided that it 
has been or can easily be established that such an object exists* Before 
students had to ^ first prove (E x)FM(x) and then use ES to get (say) 
FM(y)» LET combines these two steps into a single step* To use LET the 
student in effect types a sentence of the form: Let \v be 'such that 
FM(v).' The program will try to VERIFY (E v)FM(v) from the axioms, 
definitions, theorems,' and lines cited. If it is successful- it will 
generate" a line of the. form FM(v) (where v is noW; an ambiguous name) • 
An example of the use of LET follWs* 

Wp^ (2) pow(A,y <=: A • ^ * ' ^ / 

*2Let$ (variable) *f$ be su'ch" tTiat ^ ' ' 

(formula.) . *ini(f) and dom(f )=pow(A) and rng(f) sub A$ 
Using' *def$I NITION (Number or Name) *leq$uipollent 
• Using *def $I NITION (Number or Name) *inap$ 

Using ' *def $I NIliON (Number or Name) ^injection? . ^ 

Using *^$p ' ' ^ ' V ' 

: / ■ ■ ^' ' ' ■ ■ ■ ' ^" . ■' ■ . /' ' 

■ . n ... ' «■ ■ '\ •' ■ . . . 

. 3.2^3^5 ; Xhe REPLACE rule .. ; : . . ■ 

cv; The\ rules for replacing formulas by equivalent fqmnjlas and terms 
by equivalent terms x^ere combined into a single rule REPLACE tfhat 
replaces expression by equivalent expressions. Such generalization and 
coalescence while far from dramatic makes' the system easier to 
understand and use. ALso. the system becomes more natural in^that lof^ical 
niceties such as separate rules for replacing terms and formulas do not 
occur in^'Standard mathematical practice; there one simplj^ replaces 
equivalent expressions. . 
See the examples^j below for the details of' how -^REPLACE is used. The 



^lutro' and *Elim' in the listing of options are ^intended' as mnemonics 
. for the case in which the equivalence iis a definition* In such a case 
replacing the left hand side is eliminating the defined symbol and 
replacing the right hand side is introducing the defit^ed symbol* 

. Derive: ^ 

If A sub % & B sub C then A sub C , ' 



HYP ,(1) A sub B and B sub C 
*lrep$L ACE 

Finish,. Left (Elim), Right (Intro), or Print (F,L,R,P)? (F)*1$EFT 
Will you wish to specify? (No) 
Usink *def$I NITION . (Number or Name) *subSSET 
Occurrences (ALT MODE for all). *$. 

Finish, Left^:Elim), Right (Intro), or Print (F,L,R,P)? (F)*$ 
f 'REPLACE Using: Df. SUBSET ' 

(2) (A x)(x in A -> x in B) and (A x) (x in 3 -> x in C) 

*2vERIFY (3) *(A x)(x in A -> X in C) 
Will you wish to specify? (No) *$, . 
Using *^$0 

*3rep$L ACE 

Finish,\Lef t(Elim), Right (Intro), or Print (F,L,R,P)? (F)*r SIGHT 
Will you wish to specify? (No) *k_ . 

Using *def SI NITION . (Number or Name) *sub$S ET , ' 

Occurrences (ALT MODE for all) '*$ 



Finish, Left (Elim), Right (liitro), or Print (F,L,R,P)? (F)*$ 
3 REPLACE Using: Df. SUBSET •, ' - . 

(4) A sub C . 



Use of .REPLACE to prove, the identity condition for. ordered pairs 

■ - ■ ■ .. .... ' <i 

Derive: 

<x,y> « <u,v> iff X = u & y =^v 
HYP <1) <x;y> ^ <u,v> 

1 REPLACE Using: Df . ORDERED, Th, PAIR-IDENTITY, Df . SINGLETON, ' ' 

Th. PAIR-IDENTITY 
(2) t((x = u & X = u) or X = u & X- ^ u) 

(x = u & y = v) V X = v X y = u] - \ 

• ■■■ ^- .V ■ ' ' ^ ■ ■ 

: ((x = u & X = v) V X =» V & X = "u) 

. & . • ■ 

' (x = u & y = u) V X = u & y = u 

2 TEQ (3) X =• u and y = v - . ' ' 

TEQ (4) If X = u & y = v then <x,y> = <u,v> 

a,3 cp),4 LB ' ' . ; 

(5) . <x,y> = <ti,v> iff X = u & y '= v ' 



3.2.3.6 ZFSTART and ZFFINISH ' ^\ 

When working in a metatheory one often has to establish that , under 
given, conditions,, certain results are. proval>le in the object theory. 
One way to do this is to axiomati^e the provability relation of the 
object theory in the metatheory and to establish the result, from these 
axioms. However, it is often far easier to- simply derive the result 
directly in the object theory aiid.- then use. this fact in the metatheory. 

Such procedures were recently added to the EXCHECK system for the 
proof theory course. Two inference procedures .ari^ involved: ZFSTART~for 
starting a derivation in ZP . from the ' metatheory ; and ZFFINISH for 
finishing the derivation in ZF. and returiting to the- metatheory. After 
starting a ZF derivation from the metatb^ry you may reference prior 
results from the metatheory or . the metatheoretic part 'of the derivation. 
In the second example below^ two lines in the metatheoretic part of^e 
derivation are referenced, from the ZF part of the derivation. There is a 



restriction on the form of metatheoretlc results that tnay be referenced 
from inside ZF: they must be atomic fornmlas of "the form ZF (- F or ZF* 
|-r F. Conjunctions of such formulas are also allowed. Two simple 
examples of uses of these rules follow. 

Example 1: . r ' ,. 

Derive: . -^s ■ 

IF Y IS Z THEN ZF* |- |Y| = \Z\ 

HYP (1) Y IS Z ' . 

*2fs ST ART 

******* ****-k*^**** ZF **************** 
i f teq$ "(2) *|Yf = |Y| 
Will you wish to specify? (No) *$^ 
Using * g^0 

i ?2ff$I NISH . 

*************************************** 

2 ZFFINISH 

(3) ZF* I- |Y1 = |Y| 
*l,3teq$ (4) *.ZF* j- |Y[ = |Z|$ 
Will you wish to specify? (No) *^ 
Using *g$0 - 



Example 2; i 
Derive : 

V IF ZF I- x=y AND ZF \- y=z THEN ZF |- x=z 

'- J : ■ ^_ ■ ' ; 

WP (1) *ZF i- x=y 

^ WP ;(2); *ZF \- yfz 

*2fs$T ART 

i ***************** 2F **************** 

#l,2teq$ . (3) ' *x=2$ / ' " ^ 

Will you wish to specify? (No) *J_ 
Using *g$0 • ' • 

■ . #3'zf f SI NISH 

*************************************** 
3,1,2 Z^INISH ' ■ ' « 

..(4) ZF I- X=Z :4 




54 



Z.l.Z.l VERIFY , ^ 

The VERIFY command is designed to give the student a reasonably 
powerful method of verifying the correctness of a formula given prior 
results. For example, given A in B and B in pow(C) it is convenient ,f or ^ 
the student to be able to verify A in C simply using the definitions of 
stjbset and powerset.' The guiditig principle is that the student' should 
be doing set theory (or probability or proof theory) and not first order 
logic. Alternatively, VERIFY should be able to prove anything that is 
obvious to 'the student (and correct) within a few seconds to a few 
minutes of real time. \ . 

A simple use of the prover by' the .VERIFY command would be: ; 

1) X in A 

2) A in pow(B) : 
1^2v$ . .3) X in B_ 

USING *D$EFINITIQN (number or name) *SUB$SET' 

USING *D$EFINITION (number or name) *POW$ERSEI ^ - 

USING *G$0 

* [Failure to find a proof would .cause 'a message to be printed. ] 

■ . ■"■:■'»■-' 

The student is able to cite prior lines by number,, and pribrr^axioms, 
definitions, and theorems by; number or name .^v VERIFY at temp tis to 'use 
everything cited, and incoirporates rather few theorems implicitly, so 
the student 's Judicious choice of these prior results is essential to^ 
successful application of this command. Sinc^e^ VERIFY is intended to be 
use^i as part of an in:3grated system, see the^. section on* sample proofs 
for actual examples of i*: in use. . . V 

VERIFY .uses what is basically a resolution theorem prover. The' 
predecessor .tQ this prover -was written by " 'Vesco Marinov; see 
(Marinov, 1973). It is a level saturation prover tjjat uses, a merge 
strategy to limit the growth in length of clauses. After the first- 



rotmdy as a resolvant is generated— at least^ two of its literals must 
merge for it to be accepted. Furthermore, the- depth of terras in that 
Vclause must* be at most one greater than the deepest term input. 
Equality is dealt with via demodulation. Whenever an equality becomes, 
asserted, the simpler^ term is uniformly substituted* for the more 

'■■ ■ - ■ '■ • . •' ■ '~ ■ . ■ ' : v. " 

complicated term everywhere in the clauses being used, simplicity being; 

• ► ■ , - . . ' ^ ^ ■ . '\ ■ „ 

primarily a measure of the depth of a te^. While restrictions ; such as 

these seem at: first glance a bit severe ,"i^tiey are empirically based, an'^ 

have been chosen to. maximize the range of *'obvious" proofs obtainable in 

less than about .20 icpu seconds. Thus far, relaxing any of them^has 

dramatically reduced the number -of proofs obtainable in our domain. 

Faramodulation, f or example, is hopelessly slow for us. Surprisingly, 

the prover has been able to get more proofs in our domain without its 

set of support strategy than with it. >The proofs\we do are small enough 

that it is apparently quite efficient to simply let. forward and backward 

chaip^g meet in the middle . ^ 

Whiles' some work' has been done to explore the standard logical 

characteristics of this prover, the emphasis to date has been upon 

extending it to be flexible enough to deal with featuires such as.sorts 

•and types, fbrmula-'binding terms ,-\and answer extraction* *^ . 

■ Sorts and types present special difficulties . for' a ' metrhatiical 

prover. . For a .resolution *prover, the major effect of sorts - is to 

•restrict the unifications permitted. In bur set theory,, for example, 

-ordinals are sets and sets are general objects,' so a universally 

quantified set variable can be unified to an ordinal constant, but not 

to a general constant. : To help effect this checking of sorts, each'' 

atomic term explicitly carries a list"* of its sort and all hi.gher sorts. 



Constants are , given a sort during initializatioji, variables are 
implicitly sorted, and th^ global raechanism may note that some term t« 
not in its usual sort. (We may say that A, normally a set, is actually 
an ordinal, or specifically not an ordinal*) Thus, given a universally 
quantified variable and an atomic term, the unification algorithm need 
only see if the .s^ort of the variable appears in the list of sorts 
applicable to that term^ 

This simple scheine is complicated in three ways. First, the 
theories we use also have sort predicates. Thus, during the course of : 
ah attempted proof," we 'may . generate new sort information that permits 
unifications that were previously - blocked. The second* compli-cation 
arises in that the -sort, of a complex term depends dynamically upon the 
sorts of its sujjstituent atomic terms. *Thus^, if . an atomic term, chianges 
sort, even, during unification, itself, the sort of any complex term 
involving it may change. To dynamically sort complex terms, each 
operator is given a type during initialization, and the' sort of a 
complex term is computed each time it must be referenced. Note that to 
unify two complex terns the sorts usually need not be computed, since if 1^ 
the operators are the same and the atomic terms unify,, then, the sorts of 
the complex terms must be the, same. A third complication due to sorts 
arises when a formula is generated . similar to: 

(A x)Cx isa set -> x is an ordinal) . 

■" . . \^ ■ * 

This is very powerful information .and could dramatically speed up a 

proof if recognized, yet may yield no resolvants, since the literal 

set (A) may be suppressed as redundant. Currently such information is 

not very well used. \^ 



Another source of difficulty lies with formula binding operators 
such as abstraction (for example, '<x: x in A and x InHS}), The decision 
whether, two abstraction terms can unify may require the full power of 
^ set theory. For example, is {i: i cubed equals j cubed plus k cubed; 
iO>k integers) empty? When unifying. <x; FMl(x)> with {y: FM2(y)} a 
reasonable applrqach to attempt a proof • of (A x)(FMl(x) <-> FM2(x) )« 
' Should it succeed, the two. 'terms will unify • This strategy has the nice 
property of unifying abstraction terms- which denote the emptyset by 
virtue of having inconsistent formulas, even if those formulas are quite 
different, - ^ ' • 

A refinement upon formula binding operators' which we 'employ is a 
term and formiila binding operator, f or example the sequence- 
[i: A(i) • (i C n)] , ^ead .the set of all A sub i such tWat ± Is less than 
n. ' To unify [x: TMl(x) FMl(x)] with ly: TM2(x) FM2(x)] - the same 
strategy as above will work, except that the subproof must be of 
(A x) {sigma(FMi.(x)) <-> sigma(FM2(x)))* wher^e^rsigma is-* the most general 
unifier of TMl('x) with TM2(x). ^ 

^- r . . Bicondltionals expand into conjunctive normal form iit\?ia most 

v: ' . • . ■ . ■ ■ . ■ o- ' • * 

unfortunate way, yielding f rom T <-> the clauses (NOT P V Q) and 

■ — • . ■ V ■ ■ : 

(P V HOT Q). Thus if P is generated, Q will follow, and. then P again, 
so that many ^duplicate clauses may be generated. Since the formulas may 
be biconditiOT^als of biconditionals of schemes to avoid this 

' problem by looking at the clauses tenjd not to work.. The effects are too- 
diffuse bj^ then. By splitting proofs of biconditionals into two proofs, 
one for each conditional, we obtained an order of magnitude increase in 
speed (from 80 to 8 seconds for one typical proof ), bringing many proofs 
• below the 20 second ^time limit we impose. 




/ 3.3 Proof Theory Course 

3. 3.1 Curriculum * ' 
• Goedel's Incdmpleteness " Theorems s^re preseiyffed in the course jKf» 
. formulated, for the system ZF of set theory, the aicioms of ZF and their 
intended models (segments of the cumulative hierarchy > -are car efulj^y. 
described in/ the first p^rt of Chapter Iv In th^ second part wie -recall 
how informal ^xnathematical (in .particular, number\theoretic) notions can 
be "represented in a subsystem ZF* of set theory* (ZF* is ZF without 'the 
axiom of infinity and of the sale strength as arithmetic.) The logical 
form of these definitions is analyzed In the^ third^ part, attention is 
giverf to the problem n of .representing number theoretic, functions and 

predicates ' given by (inioraai).. recursion or induction. We show that 

■ ■ ■.'\\ . ' ■■ ■ ' ■ ' . ' ' ' • 

SIGMA-recursive functions can be introduced in a definitional extension 

of ZF*. ^ 

The informal metamathematical arguments involved in' the above 

. * . ■ 

considerations serve as the motivation for a more rigorous description 
of the syntax of ZF. That description is actually given in the first 
part of Chapter 2. In the second part of the chapter, we analyze the 
syntactic objects as binary trees ■ and formulate a theory ' for V thetn 
(analogous to Peano-arithmetic) . The theory is called TEM, and provides 
a framework f^r describing and comparing formal systems. Finally we 
indicate how syntactic notions can be presented '-in TEM. ^ ; " . • 

In Chapter 3 Goedel's First and Second .Theorems are provqd*^ — ^ 
assuming, basic representability : ^and derivability conditions. Some 
examples of nonstandard representations . of the theorem are ^iven;" they 
show that the derivability conditions a^e: crucial to the Second Theorem. 



^ ;3«3. 2 Audio Lessons in Proof Theory 

The material described above formed the- basis of the CAI course In 
proof theory. During the past yeartj,.j;bc':^OCAL language*^' was used to 
prepare lessoiis in a lecture style format owith audio. \ The display 
features of VOCAL wete particularly helpful in describing the tree-like 
structure oi. (well-founded) sets , and syntactic objects. The text of the 

• ■ Ilk ' 

lessons was presented- ^Hlrosody Mode. That is j, the text to speech step 
was achieved by concatenation of 'recorded words • The syntax of thie each 
spoken expression was also automatically analyzed, ' and ' the audio 
paraffieters . of the individual words adjusted . to 'fit •.''th:e; syntactic 
analysis* .See/(Hinckley, -et. al., 1977)._- 

3.3.3 Aut^mentation of the ' Proof Checker 

In the second year report. (Smith and Suppes, 1976) we mentioned 
that TEM was fitted straightforwardly Into the existing proof -machinery, 
and that the central results were proved on the copiputer. Yet for those 



proofs we used (in addition to the proof theoretS^c^onditions mentioned 
^above) some metamathematical rules. To dispense with the latter, 2F and 
ZF* were also implemented and ^ '"switching mechanistn' was devised for 
th^ latter of these (see Sect-.cn 3.2.3«6). This allows conceptually 
clearer derivations of the main theorems. 
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4. Teaching Initial Reading: Evaluation of Audio- .."^ ^ . 

" ■ 1 • •• ' ■ 

^•1 Letter Experyjinen^ 

- T .. ■ . . . ^ ^ 

•This* section describes one of the experiments on recognition of 
computer generated speech. In this experiment the ability of first 
graders to recognize individu^al, letter sounds was^^tested. * " 

4«1«*1 Experiraental Set^uo 

>48 f^itst graders were selected by the teachers of thre,e classes at 
the Willow Schpol in Menlo Park, California. 12 students from each 
class made up 3 treatment groups, 4 additional students from ^ach* class 
made up the control group of 12 students. Each treatment group received 
7 sessions of taped, computer generated speech and an, 8th session with, 
taped human speech. The control group receJLved 8 sessions of taped 
human speech. - , ^ * ^ ' . . - 

Tjiie sessions consisted of listening to 26 items,. each consisting of 
the carrier phrase, '"Circle the letter: followed by a letter of the 
alphabet, and after each item heard,r<'>circling the- letter name from 3 
Voices on an answer sheet. The two confusion choices came from two 
sets of letters which were used on alternate sessions.. ..Each session 

covered the alphabet without repetition, in one of eight random 

> 

order'ings. Approximately 6 seconds aifter each item was presented, the 

subjects heard a beep and the correct answer was di'"played on a flash 

, ■ ■ " ■ \. 

card. The total time between items was approxitnately ^ 9 seconds. The 
duration of the items was approximately 3 seconds, so an entire session 
took 5 to 6 minutes to present. 

The sessions wete presented in groups of two, over a two day 
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period; two sessions' each morning and two each afternoon. The first 
session in each group was followed by a short (5 nimit^) pause^ before 

beginning the next session^ At each session, the sub-'ects sat arpund a 

■ •-■■■■»'. " ^ - 

table at the head of. which was ah experimenter with flash cafda and, tape 
recorder. Each subject was supplied with an amplifier, headset » answer 
sheet and pencil* * ^ • , ^ . - ■ . ^ 

Before the first and' fifth sessions the expeTimenters introduced 
•themselves and the equipment to the subjects', and .told thera that they 
were going - to Irefar .a computer ttalk". The subjects *were informed^ that 



the experimi 



inters were interested in how well the computer talked, so 



that it was thfe computer which was being, tested, riot the subjects. They 
were then told that the- task coris.isted of' listening to each item,- 

circling the let tier ^hey lieard on the appiropriate row of /^the .answer 

. .' ■ ■ . .' if* '■ . ' ~ ■ '■ ,^ , ' 

sheetv- and look ""up a^ the flash card when they .heard* the- beep; ' They 

were told that the purpose of- the flash cards was to help them 

understand the way the computer talked. ^ ' ... 

4.1.2. - The Computer Systems for Speech Generation 
^; Four different systems for computerized speech synthesis were used 
.for this experiment and the one using- words. One system (referred to 
below as MIT) is a sophisticated phonemic synthesizer developed by 
Professors Jon Allen and Dennis Klatt at the Massachusetts Institute of 
Technology >under NSF support-. The MIT system converts text^by rule ioto 
the" control parameters for the synthesizer and thus into -speech. The. 
MIT tapes for this experiment ^ were prepared in two stages. First, ^"the! 
text was converted .into phonetic commands on Professor Allen's^ PDP-9. 
Then, these commands * were used by Dennis Klatt 's- pro$>ram on a PDP-20 



■ : ^ Vi ■,, v^>' ->-3::. .-- ■ a-. 

^. : whfch generated a digital- rei^resenlStioh . of \^h^^ sj^^ Ihis was ^ 

■ H coverted' :to an analog. ^speiecA signal by a digital to analog . converter, on . . 
> the - 1:wo other^ systems were ther VS6 and * the MLI,-. commeircial [^-^ - 
• systems produced by Votrax/ a/'divisioh of the Federal Screw Workis Inc.- 

^. Tfie VS6 . systeia was used for the letter experiment, and the more • , 



expensive ML 1 was used for the word ex^jeriment^ ^^&ese systems *'are." also 
phonemic s^thesizers, similar iti this V^pect to M^ 



.contrcjl parameters are. generated by hand, ratjier, than' by rltl^, and, - 
these* systems allow le^ control over ■ the allo'phones thea the MIT 

system. . The Votrax tapes for both exDeriments were prepared by Dr.. 

• • ■ ! " ' - • . -" ^ 

Carol. Simpson, of the Psycho-Linguistics Research Company. The fourth 

synthesis system involved in the experiment was the Micro-Intoned Speech 

Synthesis (MISS) syst^ developed, here at. the Institute under NSF' 

support (referr^ to below as LPC). It uses recorded words, which are 

digitized and then* 'compressed ?for storage on disk usins a lineap. 

predicative coding (LPC) algorithm. Sentences for the experiment, 

up of ' either a lettf<er or word tog^her with a carrier phrase were formed ■ 

by concatenation from a: ^vocabulary"' of stored words, with 'parameters 




adjusted according to a syn^otic prosody jralgorithm (developed at the • 

■ Institute-)', then expandedj:^u^ing the LPC algorithm, and converted to 'an 
analog signal." . . ' 

It should be.'emphasizi^d that we are comparing systems which are ^ 

■ ■ . ■ • ■ - ■ '• • ■ ■ . •«..■• 

* ■ . / ■ . , ; . ..... 2' ' - 

quite , different^ both in kind -arid to a certain extent in purpose. 

Neither the MIT system or the LPC system is- commercially available in 

any form whil^ the Votrax systens. aire cirrrently being- m^ 

a vastly different amount' ojE human intervention is required in the three ■ - • 
^ ' devices. .Th^ MIT system requires no human, intervention once* the text *is<** » 

. ' ^ . ■< ' . .... ^, - t: •. . 
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preseated to' the' systen; howeVer.> it does not operate in real -time • Ihe 
Yotrax systems; require a trainefd - phonetician to Manually transform text, 
into pfiohetic-xonunajnds ta 'drive "the s^rtHesiizers ■ aithoij^h some. Vtext-to- 
Votrax" connnand systems are. under developments Finally, the LPC system 
requires initial^ human interaction to record the individual' words, but 

\ • ■ ■ «• . . . 

once all the words ■ needed,, are in the vocabulary, it., converts text- -to 
speech both. automatically ajid Xn real time. Despite the fact that these 
systems, .require differing? amounts 6f human intervention : and \operate at ^ 
various speeds, they all purport . to allow computers to., talk to people' 
and it is on. that basis that we are comparing them* ^ 



4*1.3 Conor "isbn of Mean Scores for Letters 



Although the test for individual leters" in isolatj^bri is .not an easy 

.1 ■ • '■ ' ■. " ■ . .•■ ■« ur "'■'ii. ^-"v - 5 

one, the stujde'nts did -r^her well: the mean correQJt 'scores for each 

. ' ■• .■ ' •■ ' " ■ ■ - ■ ' 19" - ■ 

session were between 83 and 98 percent; see Figure 4. The variances 

were relatively large., . so there was insufficient STeparation of the. mean 

scores for session by session 'comparison.. The .difference between mean 

scores exdeeds the sum of the standard deviations in qiilyy*6 cases: 

controi^over MIX in sessions 3 ^ and 4; .. control over Votra^y^n sessi^ 

4,'6, and 7^ LPC over Votrax ia session 6. ■" ' I ■ • ^ 



We orijginally proposed.. to test also, the deit.a - modulati 
(developed and . formerly usei ':iat''. the Institute). ' However,- >the delta' 
system is not comparable in quality to' the pother systems, and by, not 
teisting^ it, we were able to increase the number/ p^f subjects .hearing the 




_ Do to the absence of / some -student who began , -the experin^fei^^^land r-rv"^ ^^^t^ 
excluSjJLon- of twd students who did nptr^ adequately respond to '.'t^fe^^^ 
and one . outlying scOTe,- some of the mekn scoressare based on less than 
the , original 12 subjects* * For the control, Tl sx:ores were used to 
compute the means f or sessions 7- and 8. For" Votf ax II-" scores were used ^ '/^ - . 
to compute the mean -for session 3, and 9 scores' for\;sessioris 5 tfirou^h •.■ - V 
^8:.; . .The means^ for MIT and LPS were computed usxn^, II • §<xcri^es \ f or all .', -.r- ' " 
'^sessions,"*. . ' • ; " * . ' " -•■ 1'^?^'-" ■ - ■ ■ ' ' -'^ ' ' 



^session number: . 




C = CONTROL, L = LPC, M-== MIT, V = VOTRAX 

■ ■ ' - - ' ■ .. ' ^ ■■ ■ ■ ' . ■' ■ 

Figure 4. Mean Scores, Let'ter Experiment, by Session- 



A taore salient aspect of the data is the regularity oyer the. first 
iseven sessions of the rank of the s-ystems. - A .sign test was -used to 
-examine this feature of the data* We began with a niill. hypothesis -tJb^t- 
the probability on any given se'ssipn of sys.teq^a having a higher mean, 
score^ thanl system hi? is one* half * We then used, ;ajbinom distribution 
to compute the probabilpLty under the null hypo thesiis. of system a^ sc 
higher than system -b^ in x sessions out of seven.' .The scores on the 
eighth session in which each class .heard/ fche recording of' the humar^ 

■ •'■ ' ^ V'' / ~' '"^^'^-v ' ■ ,■ J.:-'-' ' . " 

voice did not indicate a significant differertce between^ the four groups, 

" \ ■■ - ■:;V.^'^'^^--.--V.'-'- ■ / ■ ' ■ ■ 

so we assume, any differences, detected care the resuljt .of differences in 
the* systems used. The control group scored highest in Nseven out of 
seven sessions, Bqth MIT and LPC scored higher than Vocrax in seven 
sessions. LPC scored higher than, MIT ^in four sessions, lower in two' 



' - " *: ." ■ V ' " '. ■ ■ . .'.^ • ■ . ■ . ■ 20 

sessions, and th^ two grdups- scored the same in . one 'session* The 

\ ■ • /V ^ ' / . . - - . ■ 

results . of th^se sign tests are^ that .evidence for the superiority of the 
human voice over any of the computer systems is significant at the .OOS-- 
leyelf is evidence of. the stj(periority. x>f both LPC and MIT over 

rVot-riax. The,probabil:^ty of MIT dbing as well or better than the 4-2-1 
outcome cin the comparison to LPC, under the null hYPOthesis of equal . 
quality, is .SbS. . ^' ^ 

4.1.4' Learning Study foy Letters > \. 
The model 3elected . for study ^of t^e mean learning curve is linear 
^in .-the ctiange in^ the. probability . of. error: . \ 

where is the. prSBabiiity of error on trial n, and ^ is -the factor by 
which the. probability of error decreases itv;>^a sin.^le session. In -this 

case, q is the mean error probability during session n^ averaged over 

n^* , . . ■ , - ' , ,, . 

-students ' and letters • of jt'fee .-^alphabetV. The. parameters we heed ' to.\ 
•estimate for this, mod^ are c, and a,, since -f or any n >' Ov. q ■ -=-^^^ 

Averages of the error probabilities f or tshe first-two sessions 'Syere ." 



used as the estimates for the initial probability of error\, f 9^ each . 
^s tern, because the results of the . initial session^were affected ;by the 
•students' unf-amiliarity wi'th^^ the specif ic- task* Gl-ven q, , we then found 



^ In accordance with the assumption underlying :the> null hypothesis^ 
we assumed the probability of an equail score indicating superiority ^for 
either rsystem to be one half. Thus, an average was taken pver. the 
probability of LPC scoring* higher 4 times, - and- the probability; of ; LPC 
scqring- higher 5 times. . . 

Do to a scheduling ' confusion "on, . the first day of the letter • 
experiment, the groups hearing the MIT and LPC tapes were switched, . so 
that from session 3 through session 8', the group tljat started with- MIT 
heard instead the LPC tapes," and the original LPC ^roup . heard;^ t^ 
tapes. Since the average error probabilities over the f irst two le^jons - 
were so close (LPC: ..115, MIT: .114), and the scores on^; the s^^^ 



the ''maxiinuin likelihood e^^iate a for a^^, and & for, cr,v assuming that the > 
obsjerved valuer wer^e' normally . distributed with variance:, flf^i" about the . 
mean value: a q^. "T^e^ maximum -likelihood: estimates .'W,ere coraputed using 

• th6 remaining f ive da^ta pc^ts in* the case, .of . the computer -systems, and^' 

the -oreniiaining six points for the' control., ' ,^ 

• ;. ■ /- ■ . : ■ ' ■ ■■ ■ ■ ■ ■ ■ ■■■ * 



Table 1 



Estimated Parameter? for LineaV Model 





system 




• q. 




additional sessions (nX 
tc criterion (q < .024) 


• ■■ 


■■■ 'r' 

■ ; ■^ LPC - . 




.115 


.012 : 






■ MIT • 


.891 " 


.114 


-.006 ■ 


■ ; . 9-" .■ ... . 
















.'VOTRAX - 


.985 


.148. ■ 


.033 


115 




'.CONTROL 


•V .930 


.034 - 


.-007 





An average over the relatively stable e.rror values in the. final- six 

' ' • ' ' ' ' ■ -'^ ■■■ . ■ •, . , ■.. ^ '.■ . '-^ -.' ' .. - 

, sessions with human; speech w^s used to set, a ..criterion>for^ p 

f or the computer systems. The criterion thus computed was a "p^^obability 

of. error £. = •024. The estimates obtained for the initial probability 

^of error jand the decremental factor w^f^e then used , to compute estimates 



of ^^he nuihber of adcJitipnal- sess^ions-^Csessions- beyond the seven sessions 
used to es^^^te the initial , error rate and the decremental factor) 
needed for die, given computer systems to reach the criterion. 

The linearr learninrg model was selected for its simplicity and 

robustness.. The data seems insufficient for comparison \with a more 

" . ■ ■ . ■ , * ■ ^ ■ 

complicated 'model. A rough ^judgement of the goodness df fit of the 



session*', e^^ctly the same/ all* th LPC scores were taken to^et-her as a 7 
ses.iSlTr»-^n, and lilce wise with the MIT s'cbresf. - . 



session numb er : , 



1.00 
.99 
%98 
.97 
.96 
.95 
.94 

.,.93' 

^..92 
,^1 
.90 
.89 
.88 
.87 
.86 
.85 

• .84 

. . 82 
\-8i- 
.80 




'/ Figure 5, Predicted Scores (Learning iMpdei)- Letters 



jmo del' can be obtained 'byT cb^ observed -and predicted scores for^ 

the letter experimeat; ^ee Figure 4 and Figure 5, noting in particular ' 

■' • , ■ ' "-^ ■ r ' ' ' ' \ " ' ^ - ^•'' 

/the poor fit to' tKe Votrax datai.' Al^o of interest regarding the. fit 

the linear model is the; estimated standard deviation . of the. normal 

distribution of error about the predicted means . This'V^s^^tt^r^^l . 

^ other estimated parameters are contained in T^ble 1, which also contains 

estimates of the number' of additional sessions to criterion. Since each 

session is appTgixiQately 6 minutes long, the estimates for the number of 

sessions t<^ criterion mean, an additional 12 and 54 minutes of esqjps^re 

for L^C :and MIT to^meet the. criterion of hqgian recorded 

speech. The model ' predicts the need of ah^ additional Il«5 hours of 

exposure f or.VVotrax to meet the criterion, /however, the variance of the 



re 



Votrax estimate, fbr a .a valuexjf £ one standard deviation 



below the mean would lead to a prediction of'oinly 3»2 additional hours 
. exposure , to criterion. Also, a value of less thati . half of one 
.standard deviation greater than the mean-v/ouid imply that for Votrax, no 

learning is occurring- / / 7. . ' , 

4J1.5 Evaluation of- Specif icPrpblem Sounds^ . 

Several criteri^;^ere used in an atterapt to focus on the specif tc 
■problem letters for. -each of the systems. The^first of these was' to' 



select the letters!;, for which the sum oyer >,s^^n sessipas -of / the 
; percentage of students mistaking that lettter exceeded 67 percent • We 
• also checkedy <;7hich of ^^t letters; had no mistake-free sessions out of 

the f irst .seven. .Undei^:v;the criterion, iVbtrax had 12; Droblem 

leptersr .D^ - E,: P, J, . K^: .^^ Q, T, V, X, and Z ; 5 of which , (F, K, V,' 
yVZ) were also , problems under the second^"cricer:ion. MIT had 5 f irst 
rcriter ion: problem letters: G, G> J,; N, and Z; ^ all- be:: -C were also second 
Jcritef ion "problems. LPC ihad /4 f i^^ criterioa.problems: bV D> 0, and Z; 
of which, just Z was* arlso a second criterion pr:ob.lera. For comparison, 

the control had one first criterion problem (fJ)-, and no problems under 

the second criterion. - 

• . . " . >...■''■■ ■. ^ ■ • 

. . . . /_<■ p >T <-nt--K , 

• V - • ■ ' ■ ■ ' ' ' ' ■ ' - ■ • ■ ' ■ ' 

. \-> •;";b" < ^ -'D -nt-> G- • . ■■ 

, ■ -VOTRAX \ " ^ 

. \ K arrows .indicate direction of shift , 
nt- ~ indicated shift not testied 

.Figure 6. '.'Diap,ra'n of Stop Shifts.;:.Letter.. Experiment 
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^ The data was also examined to see if patterns of errors could be 
found /in terms of human phonetic parameters* Only in the case of the 



id Axi 1 



data^for- Votrax was there sufficient error Information to detect some ' . 

... • ^ . . • • ■ . -.^ ,v ■ ... ■ . ■ ■ . ^: 

pattierns. There we found a tendency to shift, from -unvoiced to voiced 
stops, maintaining the place of articulation, -.and a tendency ^ to shift . 
. the .place of articulation of other s^psi while maintaining the voiced 
- or unvcJiced quality. See Figure 6 foyr^ a .diagram of. these stop shifts, ■ 
and note; also in' that ' diagram the shifts from ...P • to ..E and : E.' .to ; 
paralleling -the direct jshi^t from P to B« . Other; problems :^fori V6£r^ 
were strong:, xeciprocal confusions .betwe^. K; and J, ; and between vZ arid V, 
and a strong tendency. to hear S for- the Votrax F. - 

. V All t her 'systems had. difficulty with the letter 'Z, which .was in/ . 
-general heard as V, however, only Votrax had the: relciprocal vconfuision. - 

Votrax, MIT, and t;^e control air had difficulty- with N, which -was heard - 

. " . .r ■ ■ > ■ > ; ■ ■ ■ ' -. ■ • ■.• . ^ ■ • , _ 

22 ' ' " ' » ' ' ■ "' 

as M« Both MIT and LPC had a problem "with G,- which was heard as. either 

' ' ' .'^ ' ^ ■ 

B or. D in both systems. There is evidence of vowel confusion in. the 

'j^ ■ . ' _ . - • ■ • *' 

case of the J to G shift in the MIT systemv ' . / 

' ^•Z . Word Experiment 
This section describes -the second of the experiments on recognition : 
oof computer generated speech. la this experiment the ability . of fifth 
graders to recognize initial and final consonant (or consonaitf cluster) - 
Isounds was tested. ' : . ^ 



/ " N was not a first criterion oroblem- letter for LPC, havin<^ a 
summed error percentage of 45 percent * ^ 



4.2.1 Experimental Set-up. ^ , . ^ 

fifth graders were selected by the teachers ■ of three -classes ^t 
th^ Willow School in Menlo Park, California* • 12 st3^nts from each 
^iass*. made up 3 treatment groups, 4 additional students' from each class 

made up the control group of 12^ students..^ Each treatment group received 

8 sessions/ of taped, computer generated speech and a Sth session with 
taped human speech. The control group received 9 sessions, of taped 
human '--speech • ■ " - - s ■ - ' ' 

. ' ■:■ .Table 2 ■ ■'■ 



Initial Consoftant.Soundsi with Confusion Words 



test word - confusion words 



test word ^confusion words 



I- bl 
I i>r 
Ik 
I ch 
I cl 
I cr 

I'd 
I dr. 
If' 
I fl 
I fr 
I. S 
I ?1 
■l-grv 

\ -5 

1 1 

I ta 

.1 ^: 

j p 
I pi 

I- qu 

I <r . 



bear 


"dare 


pear . 


1 


s 


sink - 


zinc 


think. 


1 


block f 


^Jrock 


clock 




sc - 


^scare 


care 


stair 


\ 


breathe 


wreathe 


sheathe 


f 


scr" screw ■ , 


threw 


stew 


1 


cash 


dash 


crash 


1 


sh 


shop 


stop 


chop 


1. 


chose 


close 


shows . . 


tsk 


skin : 


thin V 


spin 




clean > 


lean - 


seen . 


1 


.si 


^sleeve' 


leave 


weave ^ 


1 


cr^sh. 


sash . 


casrh" 


1. 


sm 


ismell'. 


swell 


fell ■ 


i 


dime 


time . " 


.lime* . . 


1 


: sn 


sniff 


^ stiff 


mtlff 


r 


drip • V 


grip ' ; 


trip 


1 


sp 


■ spark 


stark 


shark 


'■-'[.. 


fast 


cast 


past 


i 


spl 


split 


flit 


spit ' 


1 


f lo.ck 


' block 


clock 


1 


■ spr 


spring 


string 


sing ■ 


1 


free 


fee 


three 


1 


st 


sftart 


part 


-tart . 




gate 


date 


great 


1 


str 


string 


spring 


thing 


•vl 


glow 


Ipw 


go 


1 


sw 


switch : . 


which ^ 


ditch 


1 


grade 


braid • 


trade ' 


1 


t 


task , 


d^sk * 


mask 


. 1 


hand 


and- 


land 


1 


tr 


trip 


drip ■ 


strip ' 


1 • 

: 1 


jump 


''bump- 


.lump-' , 


1. 


tw 


tweed : 


weed 


,reed " . 


look 


"hook 


book** 


1 


th'^ 


those/' 


nose . 


shows 


1 


march 


arch 


starch 




th 


thi'n-^,.. 




spin 


1 


I 


fin ff- 




nice 


rice 


mice ' . ' 


1 


thr 


three - 


tree - 


spree 


1. 


post 


boast 


toast 


1 


-V 


verb 


curb — 


herb 


1 


plank 


prank 


blank 


1 


■w 


went ' . 


bent • 


^meant * 


1 . 


-price/ .. 
qilack;; - 


rice ' 0 
'crack. 


slice 

:pack: . 


1 

\ 




year . . 
zoo • • z 


ear 

.do\. 


'hear 
sue 


1 


.rasp 


'.grasp •'■ 


clasp . • 


\ 


Cth^ 


= voiced "alve^jiar 


fricative) 



Table 3 



Final Consonant Sounds, wi-tfa Con^fusion ttords, 



test word confusion' words 



test word confusion words 



lb 


tub, , 


tug 


. tough 


1 ch 


switch 


swish 


' swim 


1 d _ 


grade 


great 


.grain 


If 


- oaf 


oat 


own 


I ft 


left 


let ^ 


leg 


Is.: 


rug ^ 


rub 


•^f und 


1 j 


fudge 


fuzz 


Ik 


block 


blot 


Diond 
nest 


1 kst 


next: 


neck ' 




^11 


"or^'' ; 


.. awe 


1 lb 


. bulb 


Bulk': ' 


• bulge 


1 Xch 


nulch 


much 
-seJls 


^ muncti 


■ \ If 


self - 


■ said.^. 


1 IJ 


hulg'^ 


bulk 


t>ulb ' 


Ilk 


milk^" 


mills 


mink 


i im 


. , palm^ 


pond 


* park 


-1 Ip 


/help « 


health 


. . held ; 


1 Is 


false 


f atil;t 


fall 


lit 


fault 


•falls 


false 


r dth 


health 


help 


held 


f Iv 


delye 


dell. 


deaf 


1 m 


. dime 


dire 


dine 


■ 1 mp 


canjp 


can 't , 


cam 


In 


skin 


skiff 


skim - 


1 nch 


crunch 


crumbs 


crutch 


1 nd 


hand 


h^m » 


had^ 


1 nk 


,pla'nk 


plant 


plaid 


1 nth 


tenth 


tent 


tends 


1 nt 


went 


when 


- wet 


\ ng ^ 


spring. 


sprig 


• sprint 



I p 
I pt 

I r 

;-| rb ' 
I rch 
f rd - 
I. rf. 
I Tj 
I rk 

J . rl--; 
\ nn 

r fn 

I rs 
llrsh 



i.rt .. 
'rth 
■I rv 
I s 
I sh 
I. sk 
i sp . 
1 St 

I t : 

th 
I th~ 

I. 2 ~ 
I zh 
I (th- 



shop shot 
. rapt rack , 
. bear bend 
verb verse 

• inarch . marsh , 

bird , bir^ 



tiirf , 
3>rg^ 
spark 
curl- 
warm 
barn 
harp 
horse 
marsh ' 
start 
earth 



turn 
.lark 
spar 
' curb 
war 
^:bark - . 

raiarch 

^tarch 

earn 



starve stars 
prigie prize 



cash. 

task < 

rasp 

fast 

gate 

path 



cast 

tack 

rat 

fats 

gave 

pasis 



breathe. breeze 
gave gaze ' 
chose chore 
beige ■ bathe 
voiced^ alveolar 



shpck 
^ rat^ 

bet 

verge* 

mark 

birth 

turk, ,< ' 

lard ^ 

spot 
"^urve 

warn - 

bar 

heart 

hoard 

mark' 

Btar 
.urge 

st$rt 

pride 

camp, 
■ tax - . 

rash , . I 

fan I 

gai^ I 

•Brief ] 
^|ate f 

choke I ♦ 
"bays I 

fricative) 



I; 
I 
I 

4- 

r 
I 

.1 

4 . 



Ihe fifst^S sessions consisted^pf listening -to j 27 items, each 
-'eortsisti-n2?*=*S^'^^the car'ri^ -phrase, ''Circle the word: 'i followed by a 

monosyllabic word, and after each item circling the word heard f rory 3 

' . ■ i ' ■ • - ' ' • 

choices on an answer sheet. The taped words came from a. list of 108 

ipems (inciuding-^soine repeats -^ords. used tq cheSik *both. att''inifciai and 

Jinal' coiisonant. sound) o to x^becl^T^? initial consonants or consonant- 

^' ■ / ■ ■ > ^ --'-i-^' : . ■ - • ■• 

clusters (see Table 2), and .59 final consonsis^s or cpnsonant crijisters 

^ ( . J 



/ 
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y (see Table Zy. < Vcit list was presented twice in dif f ei^ent -^"^Mdom " 
orSteritigs: once in the first .4 sessions, again in the next four - 

. ■ .'i'' ■ ■■■ ' ■ ■ ■ i " ' ..•■■■ ■■ ■'' ./■;:■;■■ ■ • • 

• .. sessibhs; The ninth session wa's a control session , with letters, which ■ 



' used the same* ordering as- the eighth-:session of the ie^ter e^erinent', 
and was given shortly after the^ eighth' session of words* Apa:rt from 

• these differences, the experimental set-up was exactly as in the.^tter 
^ experiment; see: Sectioa 4»1»1- For a description of tKe' computer 

systems lisea see: Section 4,1*2« 

4«;2#2 ' Comparison of .. -Mean; . Scoreis for Words* - ■ 
' . \: The /score s on the word quit.e high: the ■laeati correct 

scares' ^for^^^ch session were between 78; and iOQ -.percent see Figure 
' V - 7* ^ The variances werej relatively .' large, as in the letter experiments 
V SO there was irisuf fi'cient separation of the mean scores for ^sesstbn. by . 
session comparison. The • difference' between mean scores exceeds the sum 

-of the staadard deviations in only 9 cases: control over MIT in' session ^ 

» ' *. i '•■■.>-'•' ■ 

, /*• - • 

7; control over Vot rax in all but sessions 6, 8, and 9;, LPC ov^r Votrax 
in sessions 4 and 5* A more salient aspect of the data, is the 
regularity over the fir^t eight sessions of the rank of the.systems. As. 
in the * letter ^experiijetnt a sign test was-^used for a ■ pairwise comparison 

* of the systems; see Section 1- 3 'for a description of the test. 

The control group STcored higher than Votrax and MIT in eight out of 
eight session^, and higher than LPC ^in. seven sessions:* Both MIT and LPC ' 
^ ^ . scored higher tfen Votrax in eight sessions. LPC scored higher than MIT 



' Do-tp :the- absence ? oC some- student'. who be^an* the experiment; some , 
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of' ttie mean sc^^ are based on less^than the original 12 subieces. ' For 
the control, 10 scores were used tp compute the means for s6rssions' 5 
'through 9. ;. For MIT,' 9- scores "were, used to co.mpute „the^ meatv for sessions 
. 5 -through 9.- v< ■ >• ''^^ " !* 'J .. ♦ - .. ^ ' . .'- "• ^ 

■ ■ - - . . ■ ■ -x ■ ■ 73 . .1, ■ . 



1.00. - 
.99 
,9S 
.97 
.96 
.95 
.94 
.93 
• 92 . 
.91 
.90 

'. 88 ; 

.87 
.86 
.85 
..84 
.83 
.82; . 
.81 
.80 
.79' 



session ninnber: 

2 - 3 




; : : C = CONTROL, . L = LPC; M = MIT, V = VOTRAX 

Fig-ure 7« Mean Scores, Word Experiment, by Session. 



seven sessions and lower in one sessioii-« The results of these sisn 
tests are that evidence for the superiqrity of - the human voice, LPC, and 



MIT over Vot rax is significant at the .004- l/yel^^ as is evidence of the 
superibxity -^of the human/ voice" over MIT.'V i^e evidence, for the 
superiority of the hxraan -voice over LPC, and . LPC over Mil, is* 



. { ■ 



significant -at the /.0357level. - " ' . ' 

' : / . -■ : ' •.; ' 

. ' 4.2^.3: . Evaluation of Specific' >ProSlein ' Sounds. 

: •. In order to focifs on the specific, prpblem coii^nant^ 
system, lists of /items— that nbre than x .per^^enj^^f^^t^^ 
(on. either of - the* sessions that the' item/was . t^s^ed) were^p 
« 15 , . 25y 33,. . 5(H. .ajmd 67 ;/sfee Table.^ ;4*;">>; Eacft-- list was checked t^^ .s^fe 
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what patterns of errors could be oigcerried in" terms of humaa phonetic 

. • ^ > ■ • \ > ■ ' " • ' : V 

^parameters. For the most part> the j^S percent er;;:or- range was most ^ 
amenable ta analysis. - -* ' ' ' 



Table 4- 



Number of Pr.dbiem Sounds / 



Error- Level: . 


' 15% 


•25% - 


-33% V -■ 


50% • 67% . 








initial consonants 




• VOTRAX V " 


27 


17 


13 


. 5 ■■- ' 




* MIT • 


15 


12 


I'O ' 


4 . 


.-J... 


LPC 


• 11 


- 8. V. 


5 


2 


2 . 


CONTROL 


• - ' '3 


• -1 


-\ 1 


" 1. ■ ' 


0- ' 






final 


consonan-ts 




VpTRAX 


' 35 




17 ' 


• 7 


3 ■ 


; MIT 


10 




- 6 . 


2 • 


0. 


f LPC ' 


10 


6 


2 > 


; 1 ■ ' 


0 


. . CONTROL 


3 


2 


■ ■ 1 


1- • 


0 








totals 






VOTRAX \ 


hT, 


' 43 


30 


12 


4 


'MIT 


25 


20 


16 


6 




LPC 


. 21 


* 14 . 


.7 


3 ■ 




' CONTROL ' 


■ 6 


3 - 


2 


. 2. 


0 . 



The data for LPC indicate, aside from isalated* errors, problems 
with place of articulation. ITowever, ^since. most of ti^ese pi^ce errors > . 
concern the, Xh f unvoiced theta) sound, it may be more appropriate* to say 
that LPC - has'/a place . of articulation prpbLem - with /the jth_ sound, 
confusing it with, both tiie s^ and f_ consonant sounds. Of the isolated 
ertors, the^ most' notable \ wey.e . hearing the j^Lj cluster for kw 
(orthographic 'c[> and dropping the\2. sound in ""year". r 

Most'of the problems for MIT occurre/3 in consonant clusters, rather " •• ' 

. than in individual consonant sounds. There was a tendency for* the p^, t^; 

. andVs' sounds. .to >be cftropped from initiajt.' consonant clusters beginning "- * . >, 
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P v4 > K 

I I - Int 

V . V V 

B — > D G- 
"MIT 




arrows indicate direction of .shift . . . ' 

. ' ht =^ indicated* shift not tested- ^ 

Figure 8, Diagram of Stop Shifts, Word Expetiment 

vjrith . these sounds (the effect did not appear ' notably ia^ the-*'final 
clusters). There . was . a strong tendency to shift from voiceless, to . 
voiced 'sounds, as in: £^ to b^, _t to d^, and s^ to Place of .articulation ?^ 
problems were noted in several stops and with . the th sound* It would 
appear that MIT has problems with stops and with the s* and th sounds • A 
surp^rising result was the shift from to in final cltisters where the^ - 
followed an ^ or. r^ sound. . / \ 

•^j /Votrax had problems wiph stops being dropped f rem " an 'initial 

''^ •. ^ — ^ ' ■ -.'.^ 

posicion ^in \ 4 consonant cluster, bot^^with initial and final . clusters. ; 

There were also place . of' articulation problem's with stops, v and a 

tendency to shift from unvoiced , to voiced stops. There .were also r ^ 

problems, with tli shifting to s_ and f^, as in the other systems^ aind a 

.surprising shift from th to dz\ As With MIT; there^swere problems with 

final I7 and r.1 » but they -shifted in a less surprising manner to lb ap<i> 

rd. There was also a tendency for both Jb and d[ to shift to. '^ne 

pronounced' tendency toward' recipr.6eaj^gliif-tirig i^ indicat;ive.of problems 

with the overair clarity of . Votrax^ speech, and makes an analysis in. 

terms-^ of patterns of ^ error npre diff^ some help in this regard . 

i« an texamination of the list of items j^ith error percentages .of. 50 : 



^rcent or tnot^. The errors at or above the 50 percent level would seem 

- " ' ' J ' . ^ \ ..^ - - . ^" ■ ■■■■ 

to indicate that for Votrax, place of ■articulatiqri, errors are more 

" ' *V ■ . . * ^ . - " ', " • ■ ^ - . * •j' 

pronounced than voicing errors. ^ ./ ' . . 

■■ ^ ' / ■ -.^ 1 — ^ " 

=^ In - Figure 8 we have diagramed the shifts in stops for both Votrax 
ahdr MljT. The voiceless stops hstve been placed .over the voiced stops. 



maintaining the. place of articulation. "Sounds in both the voiced ant 
voiceless- serie^ are placed with ; respect to position in the .niQuth: 



labial,' .alveolar, and ' velar. Some corroboration" of the Votrax pattern 
can be found, in Figure 6 which gives the>sane sort ^of diagram . for 
Votrax' stops in the letter e:!gperiment. . ^ v. ^ - 

Although we were . only' testing consonant sounds, there were a few 

. ^ . . • • • ^ ^ , , ■ ^ ^. . ^ - , . ^ ^ ^ 

clear voweT problems for MIT <*block' soy^iHed like ^black') -^and ^ Votrax ' 

("left' ;spurrded like "lift'):.^ ' / - * ' T ' . ' ' \ 

I ^ Si 

•* r> The, only ser;ious problem for the\^human speaker was the ^th sound 
shifting to • . - \ , * ' . „ '\ ' - ' 



41 3. Use, of Computer Getierated Speech in CAI ia Initial Readine* . 
The*^ high probability of recognition of ^oun^Ci^^sNevidenced in the 
ietter and woi*^ experiments indicates that/ some, fojfm of .-co^uter 
generated speech is adequate f or ;use in computer ai^slstedf ins^ in 
initial reading. The scores for LPC and MIT, Wh&c^h were generally well 
above 90 percent correct on. both tKe letter ind;;'Word experim.ents, is 



strong evidence of > the adequacy of theSfe syel^ems*. The - score^ for. 

n: ' \'. V- ' V , ■ ■ ■■ 

Votrax, generally - betx^een 80 and 90 percent ar^ somewhat less 

impressive*. Some . attention, howevetr'i will '"have to be paid ^ tq_ the" 

specific problem sounds .for. any.; systein chosen, "and effort made to 

provide extra practice on those items' by aj>p-ropriatel^r>^aJte.rinf, the 



curriculum. Of major inpqrtanbe in this, .r€:gard, ;xs the amount of time 

that must be spent in teaching a child xo ^understahd, the speech systeta 

as opposed to* teaching reading. shovm in Table 1» "the 

extra time *srequired by MIT artd /LPC seems sufficiently, smal^^^ The /extr^ 

time required by Vot rax, however, xnig^jt ^lead to the unpleasant result of.: 
• . • ^, ' ' '• ." ' » . . . ♦ 

'postponing or .even . exclu^ng -.tl^e teacr nr ^ words containing certain 

problem-p^one^nes. Cost analysis, will aj.. " j'^'si, % major issue- in that the . 

more adequate systems- of computer generated speech> such , as LPC and MIT 

are. still far too expensive for widespread classroom use. , ■•. ^ 




0 • 



^ 
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